2024-04-25T16:20:23.631 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T16:20:23.631 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/truncate_delay} --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262 -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/orig.config.yaml 2024-04-25T16:20:23.678 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T16:20:23.778 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/truncate_delay} email: pdonnell@redhat.com first_in_suite: false job_id: '7673262' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 client_oc_max_dirty_age: 1 debug client: 20 debug ms: 1 ms_inject_delay_max: 5 ms_inject_delay_probability: 1 ms_inject_delay_type: osd pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL39k2GKCSqtAi+vOrU1SgEJA4t9kGe6dmfHNeLhnWhpXsNdB4cWycOPWUrAUu2FkJYZPLLgZfQb0z2tPunGzMc= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ+yrv5lpOeHMsjAEWeY73vZ/jZYbu55LKVMkLHX81jDUrmrx5OV+KVQgFDC1uQQD6A8po2iFH20LBJQ2zwEktk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - exec: client.0: - cd $TESTDIR/mnt.0 && dd if=/dev/zero of=./foo count=100 - sleep 2 - cd $TESTDIR/mnt.0 && truncate --size 0 ./foo teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T16:20:23.778 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T16:20:23.779 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T16:20:23.779 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T16:20:23.780 INFO:teuthology.task.internal:Checking packages... 2024-04-25T16:20:23.804 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T16:20:23.804 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T16:20:23.804 INFO:teuthology.packaging:ref: None 2024-04-25T16:20:23.804 INFO:teuthology.packaging:tag: None 2024-04-25T16:20:23.804 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:20:23.804 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:20:23.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:20:23.942 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T16:20:23.944 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T16:20:23.950 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T16:20:23.950 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T16:20:23.965 INFO:teuthology.task.internal:Saving configuration 2024-04-25T16:20:23.982 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T16:20:23.988 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T16:20:24.013 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:14:27.052341', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL39k2GKCSqtAi+vOrU1SgEJA4t9kGe6dmfHNeLhnWhpXsNdB4cWycOPWUrAUu2FkJYZPLLgZfQb0z2tPunGzMc='} 2024-04-25T16:20:24.039 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:14:27.053672', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ+yrv5lpOeHMsjAEWeY73vZ/jZYbu55LKVMkLHX81jDUrmrx5OV+KVQgFDC1uQQD6A8po2iFH20LBJQ2zwEktk='} 2024-04-25T16:20:24.039 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T16:20:24.045 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T16:20:24.046 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T16:20:24.046 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T16:20:24.116 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3eb527bf70>, signals=[15]) 2024-04-25T16:20:24.116 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T16:20:24.122 INFO:teuthology.task.internal:Opening connections... 2024-04-25T16:20:24.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:20:24.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:20:24.199 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:20:24.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:20:24.275 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T16:20:24.281 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-25T16:20:24.299 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-25T16:20:24.300 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-25T16:20:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:20:24.358 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:20:24.358 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:20:24.358 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:20:24.358 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:20:24.358 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:20:24.358 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-25T16:20:24.383 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-25T16:20:24.401 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-25T16:20:24.402 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:20:24.459 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:20:24.460 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:20:24.460 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:20:24.460 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:20:24.460 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:20:24.460 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-25T16:20:24.474 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T16:20:24.484 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T16:20:24.491 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T16:20:24.491 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:20:24.493 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:20:24.515 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T16:20:24.521 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T16:20:24.521 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:20:24.551 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:20:24.663 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T16:20:24.716 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T16:20:24.729 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T16:20:24.729 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T16:20:24.730 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T16:20:24.730 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:20:24.730 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:20:24.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:20:24.747 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-25T16:20:24.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:20:24.748 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-25T16:20:24.805 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-437.el9.x86_64 2024-04-25T16:20:24.805 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-437.el9.x86_64 2024-04-25T16:20:24.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-04-25T16:20:24.806 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-04-25T16:20:24.806 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-437.el9.x86_64 2024-04-25T16:20:24.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-04-25T16:20:27.217 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T16:20:27.306 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-04-25T16:20:29.019 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T16:20:29.191 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T16:20:33.187 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T16:20:33.424 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-25T16:20:34.785 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-04-25T16:20:35.065 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T16:20:36.208 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T16:20:37.478 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-04-25T16:20:41.341 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-04-25T16:20:42.528 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T16:20:43.172 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T16:20:43.172 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T16:20:43.267 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T16:20:43.268 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T16:20:43.268 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T16:20:43.365 DEBUG:teuthology.orchestra.run.smithi138:> echo no | sudo yum reinstall kernel || true 2024-04-25T16:20:43.907 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 04:20:41 PM UTC. 2024-04-25T16:20:44.027 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalling: 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 255 k 2024-04-25T16:20:44.039 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 0 2024-04-25T16:20:44.040 INFO:teuthology.orchestra.run.smithi138.stderr:Operation aborted. 2024-04-25T16:20:44.092 INFO:teuthology.orchestra.run.smithi138.stdout:Is this ok [y/N]: 2024-04-25T16:20:44.093 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum reinstall -y kernel || true 2024-04-25T16:20:44.385 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T16:20:44.385 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T16:20:44.472 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T16:20:44.473 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-25T16:20:44.474 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T16:20:44.577 DEBUG:teuthology.orchestra.run.smithi094:> echo no | sudo yum reinstall kernel || true 2024-04-25T16:20:44.584 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 04:20:41 PM UTC. 2024-04-25T16:20:44.700 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalling: 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:20:44.701 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 255 k 2024-04-25T16:20:44.702 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 0 2024-04-25T16:20:44.702 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T16:20:45.068 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.2 MB/s | 255 kB 00:00 2024-04-25T16:20:45.069 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:20:45.069 INFO:teuthology.orchestra.run.smithi138.stdout:Total 693 kB/s | 255 kB 00:00 2024-04-25T16:20:45.080 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 04:20:42 PM UTC. 2024-04-25T16:20:45.090 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T16:20:45.198 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout:Reinstalling: 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T16:20:45.208 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T16:20:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:20:45.209 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 255 k 2024-04-25T16:20:45.209 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 0 2024-04-25T16:20:45.209 INFO:teuthology.orchestra.run.smithi094.stderr:Operation aborted. 2024-04-25T16:20:45.264 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T16:20:45.264 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T16:20:45.269 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T16:20:45.269 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T16:20:45.273 INFO:teuthology.orchestra.run.smithi094.stdout:Is this ok [y/N]: 2024-04-25T16:20:45.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum reinstall -y kernel || true 2024-04-25T16:20:45.382 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T16:20:45.469 INFO:teuthology.orchestra.run.smithi138.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:20:45.470 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:20:45.761 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 04:20:42 PM UTC. 2024-04-25T16:20:45.876 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T16:20:45.876 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T16:20:45.876 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-25T16:20:45.876 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout:Reinstalling: 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 255 k 2024-04-25T16:20:45.877 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 0 2024-04-25T16:20:45.878 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-25T16:20:46.081 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-25T16:20:46.081 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:20:46.081 INFO:teuthology.orchestra.run.smithi094.stdout:Total 1.2 MB/s | 255 kB 00:00 2024-04-25T16:20:46.097 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-25T16:20:46.182 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:20:46.182 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:20:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-25T16:20:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-25T16:20:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-25T16:20:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-25T16:20:46.390 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-25T16:20:46.482 INFO:teuthology.orchestra.run.smithi094.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:20:46.482 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:20:46.862 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:20:46.863 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:20:46.863 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalled: 2024-04-25T16:20:46.863 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T16:20:46.863 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:20:46.863 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T16:20:47.023 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T16:20:47.069 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T16:20:47.070 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T16:20:47.070 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T16:20:47.070 DEBUG:teuthology.parallel:result is None 2024-04-25T16:20:47.167 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:20:47.167 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:20:47.882 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:20:47.882 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:20:47.882 INFO:teuthology.orchestra.run.smithi094.stdout:Reinstalled: 2024-04-25T16:20:47.883 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T16:20:47.883 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:20:47.883 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T16:20:48.069 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T16:20:48.115 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T16:20:48.116 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T16:20:48.116 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T16:20:48.116 DEBUG:teuthology.parallel:result is None 2024-04-25T16:20:48.116 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T16:20:48.125 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T16:20:48.125 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:20:48.127 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:20:48.143 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T16:20:48.159 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T16:20:48.167 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T16:20:48.167 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:20:48.182 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:20:48.212 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T16:20:48.219 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T16:20:48.219 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:20:48.244 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:20:48.286 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:20:48.303 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:20:48.304 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:20:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:20:48.320 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T16:20:48.340 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T16:20:48.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:20:48.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:20:48.371 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T16:20:48.410 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T16:20:48.411 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:20:48.413 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:20:48.428 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:20:48.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:20:48.548 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:20:48.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:20:48.667 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:20:48.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:20:48.730 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:20:48.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:20:48.821 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:20:48.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:20:48.945 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:20:48.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:20:49.008 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-25T16:20:49.011 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-25T16:20:49.051 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:20:49.080 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:20:49.580 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T16:20:49.588 INFO:teuthology.task.internal:Starting timer... 2024-04-25T16:20:49.588 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T16:20:49.598 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T16:20:49.606 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T16:20:49.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-25T16:20:49.661 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-25T16:20:49.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-25T16:20:49.707 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-25T16:20:49.708 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T16:20:49.708 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-25T16:20:49.732 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-25T16:20:49.732 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-25T16:20:49.770 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-25T16:20:49.770 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-04-25T16:20:49.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:20:49.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:20:49.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:20:49.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:20:49.834 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T16:20:49.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:20:49.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:20:49.897 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T16:20:49.907 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T16:20:49.922 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T16:20:49.923 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T16:24:16.901 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-25T16:24:16.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:24:16.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:24:16.977 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T16:24:17.053 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:24:17.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:24:17.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:24:17.129 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T16:24:17.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:24:17.205 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T16:24:17.216 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T16:24:17.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:24:17.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:24:17.220 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:24:17.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:24:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:24:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:24:17.272 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:24:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-25T16:24:17.285 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:24:17.290 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-25T16:24:17.303 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:24:17.308 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-25T16:24:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:24:17.318 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-25T16:24:17.330 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:24:17.343 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:24:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:24:17.372 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:24:17.372 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T16:24:17.372 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.372 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.372 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.372 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.397 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:24:17.400 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:24:17.401 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T16:24:17.401 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.401 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.401 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.401 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:24:17.401 INFO:teuthology.run_tasks:Running task install... 2024-04-25T16:24:17.410 DEBUG:teuthology.task.install:project ceph 2024-04-25T16:24:17.410 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T16:24:17.410 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T16:24:17.411 INFO:teuthology.task.install:Using flavor: default 2024-04-25T16:24:17.416 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T16:24:17.416 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T16:24:17.417 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:24:17.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:24:17.419 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:24:17.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:24:17.643 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:24:17.643 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:24:17.647 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:24:17.648 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:24:17.752 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:24:17.752 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:24:17.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:24:17.777 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:24:17.777 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:24:17.815 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:24:17.815 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:24:17.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:24:17.840 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:24:17.841 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:24:17.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:24:17.906 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:24:17.914 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:24:17.977 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-25T16:24:17.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-25T16:24:17.979 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:24:18.042 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-25T16:24:18.044 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-25T16:24:18.202 INFO:teuthology.orchestra.run.smithi094.stdout:48 files removed 2024-04-25T16:24:18.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:24:18.266 INFO:teuthology.orchestra.run.smithi138.stdout:48 files removed 2024-04-25T16:24:18.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:24:18.880 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 212 kB/s | 84 kB 00:00 2024-04-25T16:24:19.010 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 182 kB/s | 84 kB 00:00 2024-04-25T16:24:19.289 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-25T16:24:19.444 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T16:24:19.690 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-25T16:24:19.753 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-04-25T16:24:20.314 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T16:24:20.426 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T16:24:22.341 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T16:24:22.754 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T16:24:26.567 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-04-25T16:24:26.879 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T16:24:28.049 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-25T16:24:28.352 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-25T16:24:29.375 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T16:24:29.590 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T16:24:34.375 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-25T16:24:34.655 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-25T16:24:36.526 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:24:36.527 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:24:36.537 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:24:36.538 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:24:36.539 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:24:36.539 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:24:36.541 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:24:36.543 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:24:36.544 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:24:36.603 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T16:24:36.612 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T16:24:36.612 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T16:24:36.612 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:24:36.613 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:24:36.614 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-25T16:24:36.615 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:24:36.616 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:24:36.617 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:24:36.618 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:24:36.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:24:36.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:24:36.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:24:36.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:24:36.623 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T16:24:36.624 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T16:24:36.625 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-04-25T16:24:36.625 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 4 Packages 2024-04-25T16:24:36.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:24:36.625 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 214 M 2024-04-25T16:24:36.625 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T16:24:36.848 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:24:36.848 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:24:36.859 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:24:36.860 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:24:36.861 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:24:36.861 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:24:36.863 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:24:36.865 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:24:36.866 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:24:36.926 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T16:24:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:24:36.935 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:24:36.936 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:24:36.937 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:24:36.938 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:24:36.939 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:24:36.940 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:24:36.941 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:24:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:24:36.943 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:24:36.944 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:24:36.945 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 4 Packages 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:24:36.946 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 214 M 2024-04-25T16:24:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-25T16:24:37.701 INFO:teuthology.orchestra.run.smithi138.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-25T16:24:37.807 INFO:teuthology.orchestra.run.smithi094.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T16:24:38.360 INFO:teuthology.orchestra.run.smithi138.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 863 kB 00:00 2024-04-25T16:24:38.565 INFO:teuthology.orchestra.run.smithi094.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 863 kB 00:00 2024-04-25T16:24:38.576 INFO:teuthology.orchestra.run.smithi138.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 679 kB/s | 147 kB 00:00 2024-04-25T16:24:38.674 INFO:teuthology.orchestra.run.smithi094.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.3 MB/s | 147 kB 00:00 2024-04-25T16:24:38.810 INFO:teuthology.orchestra.run.smithi138.stdout:(4/152): ceph-common-19.0.0-3291.g994e2f20.el9. 15 MB/s | 20 MB 00:01 2024-04-25T16:24:39.166 INFO:teuthology.orchestra.run.smithi094.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.6 MB/s | 2.3 MB 00:00 2024-04-25T16:24:39.258 INFO:teuthology.orchestra.run.smithi094.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-04-25T16:24:39.268 INFO:teuthology.orchestra.run.smithi138.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.0 MB/s | 5.4 MB 00:01 2024-04-25T16:24:39.311 INFO:teuthology.orchestra.run.smithi138.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-04-25T16:24:39.403 INFO:teuthology.orchestra.run.smithi138.stdout:(7/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.7 MB/s | 2.3 MB 00:00 2024-04-25T16:24:39.425 INFO:teuthology.orchestra.run.smithi094.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-25T16:24:39.917 INFO:teuthology.orchestra.run.smithi094.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 7.4 MB/s | 4.9 MB 00:00 2024-04-25T16:24:39.986 INFO:teuthology.orchestra.run.smithi138.stdout:(8/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 6.8 MB/s | 4.9 MB 00:00 2024-04-25T16:24:40.103 INFO:teuthology.orchestra.run.smithi138.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 216 kB/s | 25 kB 00:00 2024-04-25T16:24:41.967 INFO:teuthology.orchestra.run.smithi094.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 4.9 MB/s | 10 MB 00:02 2024-04-25T16:24:42.642 INFO:teuthology.orchestra.run.smithi094.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 37 kB/s | 25 kB 00:00 2024-04-25T16:24:42.986 INFO:teuthology.orchestra.run.smithi138.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 4.8 MB/s | 18 MB 00:03 2024-04-25T16:24:43.095 INFO:teuthology.orchestra.run.smithi138.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.0 MB/s | 224 kB 00:00 2024-04-25T16:24:43.179 INFO:teuthology.orchestra.run.smithi138.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 325 kB/s | 27 kB 00:00 2024-04-25T16:24:43.262 INFO:teuthology.orchestra.run.smithi138.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 380 kB/s | 32 kB 00:00 2024-04-25T16:24:43.492 INFO:teuthology.orchestra.run.smithi094.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 4.4 MB/s | 18 MB 00:04 2024-04-25T16:24:43.504 INFO:teuthology.orchestra.run.smithi138.stdout:(14/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.5 MB/s | 10 MB 00:04 2024-04-25T16:24:43.538 INFO:teuthology.orchestra.run.smithi138.stdout:(15/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.7 MB/s | 756 kB 00:00 2024-04-25T16:24:43.551 INFO:teuthology.orchestra.run.smithi094.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 3.8 MB/s | 224 kB 00:00 2024-04-25T16:24:43.596 INFO:teuthology.orchestra.run.smithi138.stdout:(16/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T16:24:43.630 INFO:teuthology.orchestra.run.smithi138.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.3 MB/s | 127 kB 00:00 2024-04-25T16:24:43.634 INFO:teuthology.orchestra.run.smithi094.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 324 kB/s | 27 kB 00:00 2024-04-25T16:24:43.780 INFO:teuthology.orchestra.run.smithi138.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 2.6 MB/s | 497 kB 00:00 2024-04-25T16:24:44.118 INFO:teuthology.orchestra.run.smithi094.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 66 kB/s | 32 kB 00:00 2024-04-25T16:24:44.156 INFO:teuthology.orchestra.run.smithi138.stdout:(19/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 9.8 MB/s | 5.2 MB 00:00 2024-04-25T16:24:44.189 INFO:teuthology.orchestra.run.smithi138.stdout:(20/152): python3-ceph-argparse-19.0.0-3291.g99 110 kB/s | 45 kB 00:00 2024-04-25T16:24:44.240 INFO:teuthology.orchestra.run.smithi138.stdout:(21/152): python3-ceph-common-19.0.0-3291.g994e 1.6 MB/s | 135 kB 00:00 2024-04-25T16:24:44.260 INFO:teuthology.orchestra.run.smithi094.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 5.2 MB/s | 756 kB 00:00 2024-04-25T16:24:44.307 INFO:teuthology.orchestra.run.smithi138.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 4.7 MB/s | 321 kB 00:00 2024-04-25T16:24:44.335 INFO:teuthology.orchestra.run.smithi094.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.1 MB/s | 158 kB 00:00 2024-04-25T16:24:44.365 INFO:teuthology.orchestra.run.smithi138.stdout:(23/152): python3-cephfs-19.0.0-3291.g994e2f20. 919 kB/s | 161 kB 00:00 2024-04-25T16:24:44.399 INFO:teuthology.orchestra.run.smithi138.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T16:24:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.9 MB/s | 127 kB 00:00 2024-04-25T16:24:44.457 INFO:teuthology.orchestra.run.smithi138.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.1 MB/s | 100 kB 00:00 2024-04-25T16:24:44.483 INFO:teuthology.orchestra.run.smithi138.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 86 kB 00:00 2024-04-25T16:24:44.520 INFO:teuthology.orchestra.run.smithi094.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 4.2 MB/s | 497 kB 00:00 2024-04-25T16:24:44.558 INFO:teuthology.orchestra.run.smithi138.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.2 MB/s | 169 kB 00:00 2024-04-25T16:24:44.684 INFO:teuthology.orchestra.run.smithi138.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 273 kB/s | 34 kB 00:00 2024-04-25T16:24:44.759 INFO:teuthology.orchestra.run.smithi138.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.9 MB/s | 142 kB 00:00 2024-04-25T16:24:45.226 INFO:teuthology.orchestra.run.smithi138.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 7.6 MB/s | 3.5 MB 00:00 2024-04-25T16:24:45.287 INFO:teuthology.orchestra.run.smithi094.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.7 MB/s | 5.2 MB 00:00 2024-04-25T16:24:45.354 INFO:teuthology.orchestra.run.smithi094.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 681 kB/s | 45 kB 00:00 2024-04-25T16:24:45.429 INFO:teuthology.orchestra.run.smithi094.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.8 MB/s | 135 kB 00:00 2024-04-25T16:24:45.596 INFO:teuthology.orchestra.run.smithi094.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 966 kB/s | 161 kB 00:00 2024-04-25T16:24:45.680 INFO:teuthology.orchestra.run.smithi094.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.8 MB/s | 321 kB 00:00 2024-04-25T16:24:45.764 INFO:teuthology.orchestra.run.smithi094.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.5 MB/s | 298 kB 00:00 2024-04-25T16:24:45.831 INFO:teuthology.orchestra.run.smithi094.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T16:24:45.881 INFO:teuthology.orchestra.run.smithi094.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 86 kB 00:00 2024-04-25T16:24:45.893 INFO:teuthology.orchestra.run.smithi138.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-25T16:24:45.943 INFO:teuthology.orchestra.run.smithi138.stdout:(32/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 2.0 MB/s | 3.0 MB 00:01 2024-04-25T16:24:45.994 INFO:teuthology.orchestra.run.smithi138.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.4 MB/s | 249 kB 00:00 2024-04-25T16:24:46.036 INFO:teuthology.orchestra.run.smithi138.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 535 kB/s | 49 kB 00:00 2024-04-25T16:24:46.053 INFO:teuthology.orchestra.run.smithi138.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 280 kB/s | 16 kB 00:00 2024-04-25T16:24:46.211 INFO:teuthology.orchestra.run.smithi138.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T16:24:46.262 INFO:teuthology.orchestra.run.smithi138.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 366 kB/s | 18 kB 00:00 2024-04-25T16:24:46.287 INFO:teuthology.orchestra.run.smithi138.stdout:(38/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.1 MB/s | 273 kB 00:00 2024-04-25T16:24:46.315 INFO:teuthology.orchestra.run.smithi094.stdout:(26/152): ceph-common-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 20 MB 00:08 2024-04-25T16:24:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:(27/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 6.2 MB/s | 3.0 MB 00:00 2024-04-25T16:24:46.424 INFO:teuthology.orchestra.run.smithi094.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.5 MB/s | 169 kB 00:00 2024-04-25T16:24:46.441 INFO:teuthology.orchestra.run.smithi094.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 512 kB/s | 34 kB 00:00 2024-04-25T16:24:46.454 INFO:teuthology.orchestra.run.smithi138.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-04-25T16:24:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.8 MB/s | 142 kB 00:00 2024-04-25T16:24:46.504 INFO:teuthology.orchestra.run.smithi138.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-25T16:24:46.529 INFO:teuthology.orchestra.run.smithi138.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-25T16:24:46.572 INFO:teuthology.orchestra.run.smithi138.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-25T16:24:46.638 INFO:teuthology.orchestra.run.smithi138.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T16:24:46.664 INFO:teuthology.orchestra.run.smithi138.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T16:24:46.689 INFO:teuthology.orchestra.run.smithi138.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T16:24:46.706 INFO:teuthology.orchestra.run.smithi138.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T16:24:46.731 INFO:teuthology.orchestra.run.smithi138.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T16:24:46.757 INFO:teuthology.orchestra.run.smithi138.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T16:24:46.767 INFO:teuthology.orchestra.run.smithi094.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 11 MB/s | 3.5 MB 00:00 2024-04-25T16:24:46.799 INFO:teuthology.orchestra.run.smithi138.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T16:24:46.824 INFO:teuthology.orchestra.run.smithi138.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T16:24:46.825 INFO:teuthology.orchestra.run.smithi094.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 4.2 MB/s | 249 kB 00:00 2024-04-25T16:24:46.850 INFO:teuthology.orchestra.run.smithi138.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T16:24:46.874 INFO:teuthology.orchestra.run.smithi138.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T16:24:46.876 INFO:teuthology.orchestra.run.smithi094.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 988 kB/s | 49 kB 00:00 2024-04-25T16:24:46.899 INFO:teuthology.orchestra.run.smithi138.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-04-25T16:24:46.926 INFO:teuthology.orchestra.run.smithi094.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 330 kB/s | 16 kB 00:00 2024-04-25T16:24:46.985 INFO:teuthology.orchestra.run.smithi094.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.6 MB/s | 273 kB 00:00 2024-04-25T16:24:47.000 INFO:teuthology.orchestra.run.smithi138.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T16:24:47.025 INFO:teuthology.orchestra.run.smithi138.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 18 kB 00:00 2024-04-25T16:24:47.059 INFO:teuthology.orchestra.run.smithi138.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-04-25T16:24:47.084 INFO:teuthology.orchestra.run.smithi138.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T16:24:47.109 INFO:teuthology.orchestra.run.smithi138.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-04-25T16:24:47.134 INFO:teuthology.orchestra.run.smithi138.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T16:24:47.143 INFO:teuthology.orchestra.run.smithi094.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T16:24:47.168 INFO:teuthology.orchestra.run.smithi138.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T16:24:47.194 INFO:teuthology.orchestra.run.smithi094.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 367 kB/s | 18 kB 00:00 2024-04-25T16:24:47.194 INFO:teuthology.orchestra.run.smithi138.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 438 kB/s | 25 kB 00:00 2024-04-25T16:24:47.219 INFO:teuthology.orchestra.run.smithi138.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-04-25T16:24:47.244 INFO:teuthology.orchestra.run.smithi138.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T16:24:47.270 INFO:teuthology.orchestra.run.smithi138.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T16:24:47.287 INFO:teuthology.orchestra.run.smithi138.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 847 kB/s | 35 kB 00:00 2024-04-25T16:24:47.302 INFO:teuthology.orchestra.run.smithi094.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-25T16:24:47.412 INFO:teuthology.orchestra.run.smithi138.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-25T16:24:47.478 INFO:teuthology.orchestra.run.smithi094.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-25T16:24:47.504 INFO:teuthology.orchestra.run.smithi138.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-25T16:24:47.537 INFO:teuthology.orchestra.run.smithi094.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T16:24:47.546 INFO:teuthology.orchestra.run.smithi138.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T16:24:47.580 INFO:teuthology.orchestra.run.smithi138.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T16:24:47.605 INFO:teuthology.orchestra.run.smithi138.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T16:24:47.695 INFO:teuthology.orchestra.run.smithi094.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-25T16:24:47.772 INFO:teuthology.orchestra.run.smithi138.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T16:24:47.814 INFO:teuthology.orchestra.run.smithi138.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-25T16:24:47.829 INFO:teuthology.orchestra.run.smithi094.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-25T16:24:47.840 INFO:teuthology.orchestra.run.smithi138.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T16:24:47.887 INFO:teuthology.orchestra.run.smithi094.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T16:24:47.906 INFO:teuthology.orchestra.run.smithi138.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T16:24:48.098 INFO:teuthology.orchestra.run.smithi138.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-25T16:24:48.131 INFO:teuthology.orchestra.run.smithi138.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-04-25T16:24:48.165 INFO:teuthology.orchestra.run.smithi138.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 577 kB/s | 19 kB 00:00 2024-04-25T16:24:48.190 INFO:teuthology.orchestra.run.smithi138.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-04-25T16:24:48.216 INFO:teuthology.orchestra.run.smithi138.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T16:24:48.241 INFO:teuthology.orchestra.run.smithi138.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T16:24:48.321 INFO:teuthology.orchestra.run.smithi094.stdout:(44/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 7.6 MB/s | 43 MB 00:05 2024-04-25T16:24:48.346 INFO:teuthology.orchestra.run.smithi094.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 295 kB/s | 135 kB 00:00 2024-04-25T16:24:48.363 INFO:teuthology.orchestra.run.smithi094.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T16:24:48.375 INFO:teuthology.orchestra.run.smithi138.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T16:24:48.388 INFO:teuthology.orchestra.run.smithi094.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T16:24:48.408 INFO:teuthology.orchestra.run.smithi138.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-25T16:24:48.434 INFO:teuthology.orchestra.run.smithi138.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T16:24:48.459 INFO:teuthology.orchestra.run.smithi138.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T16:24:48.484 INFO:teuthology.orchestra.run.smithi138.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-25T16:24:48.489 INFO:teuthology.orchestra.run.smithi094.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 3.7 MB/s | 7.4 MB 00:01 2024-04-25T16:24:48.509 INFO:teuthology.orchestra.run.smithi138.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T16:24:48.531 INFO:teuthology.orchestra.run.smithi094.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-25T16:24:48.534 INFO:teuthology.orchestra.run.smithi138.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T16:24:48.560 INFO:teuthology.orchestra.run.smithi138.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T16:24:48.565 INFO:teuthology.orchestra.run.smithi094.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-04-25T16:24:48.585 INFO:teuthology.orchestra.run.smithi138.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T16:24:48.590 INFO:teuthology.orchestra.run.smithi094.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-25T16:24:48.607 INFO:teuthology.orchestra.run.smithi094.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-04-25T16:24:48.611 INFO:teuthology.orchestra.run.smithi138.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T16:24:48.632 INFO:teuthology.orchestra.run.smithi094.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-25T16:24:48.649 INFO:teuthology.orchestra.run.smithi094.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 555 kB/s | 33 kB 00:00 2024-04-25T16:24:48.653 INFO:teuthology.orchestra.run.smithi138.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-25T16:24:48.691 INFO:teuthology.orchestra.run.smithi094.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T16:24:48.708 INFO:teuthology.orchestra.run.smithi094.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T16:24:48.711 INFO:teuthology.orchestra.run.smithi138.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 789 kB/s | 44 kB 00:00 2024-04-25T16:24:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-25T16:24:48.770 INFO:teuthology.orchestra.run.smithi138.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-04-25T16:24:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-25T16:24:48.818 INFO:teuthology.orchestra.run.smithi094.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 431 kB/s | 45 kB 00:00 2024-04-25T16:24:48.837 INFO:teuthology.orchestra.run.smithi138.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 558 kB/s | 36 kB 00:00 2024-04-25T16:24:48.843 INFO:teuthology.orchestra.run.smithi094.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-25T16:24:48.868 INFO:teuthology.orchestra.run.smithi094.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-04-25T16:24:48.894 INFO:teuthology.orchestra.run.smithi094.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T16:24:48.919 INFO:teuthology.orchestra.run.smithi094.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T16:24:48.952 INFO:teuthology.orchestra.run.smithi094.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T16:24:48.978 INFO:teuthology.orchestra.run.smithi094.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 609 kB/s | 35 kB 00:00 2024-04-25T16:24:49.003 INFO:teuthology.orchestra.run.smithi094.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-25T16:24:49.021 INFO:teuthology.orchestra.run.smithi138.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-25T16:24:49.046 INFO:teuthology.orchestra.run.smithi138.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 230 kB/s | 47 kB 00:00 2024-04-25T16:24:49.088 INFO:teuthology.orchestra.run.smithi138.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-25T16:24:49.113 INFO:teuthology.orchestra.run.smithi138.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 861 kB/s | 58 kB 00:00 2024-04-25T16:24:49.138 INFO:teuthology.orchestra.run.smithi138.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T16:24:49.145 INFO:teuthology.orchestra.run.smithi094.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-25T16:24:49.164 INFO:teuthology.orchestra.run.smithi138.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T16:24:49.179 INFO:teuthology.orchestra.run.smithi094.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.2 MB/s | 1.6 MB 00:00 2024-04-25T16:24:49.189 INFO:teuthology.orchestra.run.smithi138.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-25T16:24:49.237 INFO:teuthology.orchestra.run.smithi094.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T16:24:49.263 INFO:teuthology.orchestra.run.smithi094.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-25T16:24:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T16:24:49.330 INFO:teuthology.orchestra.run.smithi094.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T16:24:49.355 INFO:teuthology.orchestra.run.smithi094.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T16:24:49.389 INFO:teuthology.orchestra.run.smithi138.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-04-25T16:24:49.456 INFO:teuthology.orchestra.run.smithi094.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T16:24:49.498 INFO:teuthology.orchestra.run.smithi094.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-25T16:24:49.514 INFO:teuthology.orchestra.run.smithi138.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-04-25T16:24:49.582 INFO:teuthology.orchestra.run.smithi094.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-04-25T16:24:49.598 INFO:teuthology.orchestra.run.smithi138.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.6 MB/s | 548 kB 00:00 2024-04-25T16:24:49.607 INFO:teuthology.orchestra.run.smithi094.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T16:24:49.632 INFO:teuthology.orchestra.run.smithi094.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 195 kB/s | 26 kB 00:00 2024-04-25T16:24:49.648 INFO:teuthology.orchestra.run.smithi138.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-25T16:24:49.657 INFO:teuthology.orchestra.run.smithi094.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-25T16:24:49.683 INFO:teuthology.orchestra.run.smithi094.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-25T16:24:49.690 INFO:teuthology.orchestra.run.smithi138.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-25T16:24:49.708 INFO:teuthology.orchestra.run.smithi094.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T16:24:49.733 INFO:teuthology.orchestra.run.smithi094.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-25T16:24:49.741 INFO:teuthology.orchestra.run.smithi138.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-04-25T16:24:49.767 INFO:teuthology.orchestra.run.smithi094.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-25T16:24:49.791 INFO:teuthology.orchestra.run.smithi138.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-04-25T16:24:49.834 INFO:teuthology.orchestra.run.smithi094.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T16:24:49.841 INFO:teuthology.orchestra.run.smithi138.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T16:24:49.859 INFO:teuthology.orchestra.run.smithi094.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 626 kB/s | 77 kB 00:00 2024-04-25T16:24:49.885 INFO:teuthology.orchestra.run.smithi094.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 366 kB/s | 42 kB 00:00 2024-04-25T16:24:49.908 INFO:teuthology.orchestra.run.smithi138.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T16:24:49.909 INFO:teuthology.orchestra.run.smithi094.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T16:24:49.943 INFO:teuthology.orchestra.run.smithi094.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-04-25T16:24:49.968 INFO:teuthology.orchestra.run.smithi094.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T16:24:49.992 INFO:teuthology.orchestra.run.smithi138.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T16:24:49.994 INFO:teuthology.orchestra.run.smithi094.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 614 kB/s | 51 kB 00:00 2024-04-25T16:24:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T16:24:50.045 INFO:teuthology.orchestra.run.smithi094.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 825 kB/s | 42 kB 00:00 2024-04-25T16:24:50.067 INFO:teuthology.orchestra.run.smithi138.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T16:24:50.070 INFO:teuthology.orchestra.run.smithi094.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 894 kB/s | 44 kB 00:00 2024-04-25T16:24:50.093 INFO:teuthology.orchestra.run.smithi138.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 838 kB 00:00 2024-04-25T16:24:50.104 INFO:teuthology.orchestra.run.smithi094.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-04-25T16:24:50.118 INFO:teuthology.orchestra.run.smithi138.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-04-25T16:24:50.120 INFO:teuthology.orchestra.run.smithi094.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 748 kB/s | 36 kB 00:00 2024-04-25T16:24:50.134 INFO:teuthology.orchestra.run.smithi138.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 770 kB/s | 32 kB 00:00 2024-04-25T16:24:50.146 INFO:teuthology.orchestra.run.smithi094.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.2 MB/s | 47 kB 00:00 2024-04-25T16:24:50.152 INFO:teuthology.orchestra.run.smithi138.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 444 kB/s | 15 kB 00:00 2024-04-25T16:24:50.188 INFO:teuthology.orchestra.run.smithi094.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.6 MB/s | 305 kB 00:00 2024-04-25T16:24:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-25T16:24:50.210 INFO:teuthology.orchestra.run.smithi138.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T16:24:50.236 INFO:teuthology.orchestra.run.smithi138.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T16:24:50.247 INFO:teuthology.orchestra.run.smithi094.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T16:24:50.264 INFO:teuthology.orchestra.run.smithi094.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T16:24:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-04-25T16:24:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 224 kB/s | 45 kB 00:00 2024-04-25T16:24:50.473 INFO:teuthology.orchestra.run.smithi094.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 107 kB/s | 22 kB 00:00 2024-04-25T16:24:50.499 INFO:teuthology.orchestra.run.smithi094.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 17 MB/s | 863 kB 00:00 2024-04-25T16:24:50.607 INFO:teuthology.orchestra.run.smithi094.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-25T16:24:50.628 INFO:teuthology.orchestra.run.smithi138.stdout:(119/152): ceph-test-19.0.0-3291.g994e2f20.el9. 4.1 MB/s | 43 MB 00:10 2024-04-25T16:24:50.653 INFO:teuthology.orchestra.run.smithi138.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 85 kB/s | 35 kB 00:00 2024-04-25T16:24:50.666 INFO:teuthology.orchestra.run.smithi094.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-25T16:24:50.687 INFO:teuthology.orchestra.run.smithi138.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 637 kB/s | 303 kB 00:00 2024-04-25T16:24:50.725 INFO:teuthology.orchestra.run.smithi094.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-25T16:24:50.737 INFO:teuthology.orchestra.run.smithi138.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-25T16:24:50.750 INFO:teuthology.orchestra.run.smithi094.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-04-25T16:24:50.779 INFO:teuthology.orchestra.run.smithi138.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-25T16:24:50.792 INFO:teuthology.orchestra.run.smithi094.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-04-25T16:24:50.812 INFO:teuthology.orchestra.run.smithi138.stdout:(124/152): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-04-25T16:24:50.817 INFO:teuthology.orchestra.run.smithi094.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T16:24:50.838 INFO:teuthology.orchestra.run.smithi138.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-25T16:24:50.863 INFO:teuthology.orchestra.run.smithi138.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-25T16:24:50.868 INFO:teuthology.orchestra.run.smithi094.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T16:24:50.888 INFO:teuthology.orchestra.run.smithi138.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-25T16:24:50.914 INFO:teuthology.orchestra.run.smithi138.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-25T16:24:50.926 INFO:teuthology.orchestra.run.smithi094.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-04-25T16:24:50.964 INFO:teuthology.orchestra.run.smithi138.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 931 kB/s | 46 kB 00:00 2024-04-25T16:24:50.994 INFO:teuthology.orchestra.run.smithi094.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T16:24:51.006 INFO:teuthology.orchestra.run.smithi138.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-04-25T16:24:51.019 INFO:teuthology.orchestra.run.smithi094.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-04-25T16:24:51.031 INFO:teuthology.orchestra.run.smithi138.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-25T16:24:51.052 INFO:teuthology.orchestra.run.smithi094.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-25T16:24:51.056 INFO:teuthology.orchestra.run.smithi138.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T16:24:51.078 INFO:teuthology.orchestra.run.smithi094.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-25T16:24:51.090 INFO:teuthology.orchestra.run.smithi138.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-04-25T16:24:51.103 INFO:teuthology.orchestra.run.smithi094.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-04-25T16:24:51.115 INFO:teuthology.orchestra.run.smithi138.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-25T16:24:51.128 INFO:teuthology.orchestra.run.smithi094.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T16:24:51.140 INFO:teuthology.orchestra.run.smithi138.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T16:24:51.166 INFO:teuthology.orchestra.run.smithi138.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T16:24:51.172 INFO:teuthology.orchestra.run.smithi094.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.7 MB/s | 358 kB 00:00 2024-04-25T16:24:51.195 INFO:teuthology.orchestra.run.smithi094.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-04-25T16:24:51.199 INFO:teuthology.orchestra.run.smithi138.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-25T16:24:51.221 INFO:teuthology.orchestra.run.smithi094.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T16:24:51.224 INFO:teuthology.orchestra.run.smithi138.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2024-04-25T16:24:51.250 INFO:teuthology.orchestra.run.smithi138.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T16:24:51.255 INFO:teuthology.orchestra.run.smithi094.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T16:24:51.275 INFO:teuthology.orchestra.run.smithi138.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-04-25T16:24:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-25T16:24:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-25T16:24:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T16:24:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T16:24:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-25T16:24:51.413 INFO:teuthology.orchestra.run.smithi094.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-25T16:24:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T16:24:51.447 INFO:teuthology.orchestra.run.smithi094.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-25T16:24:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T16:24:51.481 INFO:teuthology.orchestra.run.smithi094.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-25T16:24:51.506 INFO:teuthology.orchestra.run.smithi094.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-04-25T16:24:51.541 INFO:teuthology.orchestra.run.smithi094.stdout:(127/152): python3-grpcio-tools-1.46.7-10.el9.x 453 kB/s | 144 kB 00:00 2024-04-25T16:24:51.556 INFO:teuthology.orchestra.run.smithi094.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T16:24:51.574 INFO:teuthology.orchestra.run.smithi094.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 291 kB/s | 20 kB 00:00 2024-04-25T16:24:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-25T16:24:51.599 INFO:teuthology.orchestra.run.smithi094.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-25T16:24:51.645 INFO:teuthology.orchestra.run.smithi138.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-04-25T16:24:51.649 INFO:teuthology.orchestra.run.smithi094.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T16:24:51.666 INFO:teuthology.orchestra.run.smithi094.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-04-25T16:24:51.692 INFO:teuthology.orchestra.run.smithi094.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 670 kB/s | 79 kB 00:00 2024-04-25T16:24:51.717 INFO:teuthology.orchestra.run.smithi094.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-04-25T16:24:51.734 INFO:teuthology.orchestra.run.smithi094.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-04-25T16:24:51.745 INFO:teuthology.orchestra.run.smithi138.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T16:24:51.759 INFO:teuthology.orchestra.run.smithi094.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T16:24:51.784 INFO:teuthology.orchestra.run.smithi094.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-04-25T16:24:51.818 INFO:teuthology.orchestra.run.smithi094.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-25T16:24:51.860 INFO:teuthology.orchestra.run.smithi094.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T16:24:51.886 INFO:teuthology.orchestra.run.smithi094.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-04-25T16:24:51.911 INFO:teuthology.orchestra.run.smithi094.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 469 kB/s | 59 kB 00:00 2024-04-25T16:24:51.936 INFO:teuthology.orchestra.run.smithi094.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-25T16:24:51.962 INFO:teuthology.orchestra.run.smithi094.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T16:24:51.987 INFO:teuthology.orchestra.run.smithi094.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T16:24:52.012 INFO:teuthology.orchestra.run.smithi094.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-25T16:24:52.038 INFO:teuthology.orchestra.run.smithi094.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T16:24:52.063 INFO:teuthology.orchestra.run.smithi094.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T16:24:52.163 INFO:teuthology.orchestra.run.smithi094.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T16:24:52.280 INFO:teuthology.orchestra.run.smithi094.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T16:24:52.362 INFO:teuthology.orchestra.run.smithi138.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-04-25T16:24:52.447 INFO:teuthology.orchestra.run.smithi094.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T16:24:52.495 INFO:teuthology.orchestra.run.smithi138.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 3.1 MB/s | 3.3 MB 00:01 2024-04-25T16:24:52.501 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:24:52.502 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 214 MB 00:15 2024-04-25T16:24:52.506 INFO:teuthology.orchestra.run.smithi094.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 7.1 MB/s | 3.3 MB 00:00 2024-04-25T16:24:52.606 INFO:teuthology.orchestra.run.smithi094.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-25T16:24:52.610 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:24:52.611 INFO:teuthology.orchestra.run.smithi094.stdout:Total 14 MB/s | 214 MB 00:15 2024-04-25T16:24:53.574 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T16:24:53.718 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T16:24:53.718 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T16:24:53.721 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-25T16:24:53.863 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-25T16:24:53.864 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-25T16:24:55.435 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T16:24:55.435 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T16:24:55.583 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-25T16:24:55.585 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-25T16:25:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-25T16:25:01.483 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T16:25:01.536 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:25:01.566 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:25:01.574 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:25:01.604 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:25:01.627 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:25:01.658 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:25:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:25:01.688 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:25:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:25:01.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:25:01.730 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:25:01.759 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:25:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:25:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:25:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:25:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:25:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:25:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:25:02.349 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:25:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:25:02.435 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:25:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:25:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:25:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:25:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:25:02.655 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:25:02.667 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:25:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:25:02.704 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:25:02.732 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:25:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:25:02.822 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:25:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:25:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:25:02.883 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:25:02.906 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:25:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:25:02.928 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:25:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:25:02.957 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:25:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:25:03.127 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:25:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:25:03.159 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:25:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:25:03.267 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:25:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:25:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:25:03.360 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:25:03.400 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:25:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:25:03.434 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:25:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:25:03.514 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:25:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:25:03.563 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:25:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:25:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:25:03.619 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:25:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:25:03.650 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:25:03.686 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:25:03.708 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:25:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:25:03.769 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:25:03.799 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:25:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:25:03.843 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:25:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:25:03.892 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:25:03.921 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:25:03.955 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:25:03.969 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:25:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:25:04.003 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:25:04.026 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:25:04.046 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:25:04.074 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:25:04.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:25:04.144 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:25:04.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:25:04.190 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:25:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:25:04.264 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:25:04.373 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:25:04.399 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:25:04.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:25:04.459 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:25:04.950 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:25:04.985 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:25:05.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:25:05.068 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:25:05.079 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:25:05.111 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:25:05.117 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:25:05.149 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:25:05.152 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:25:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:25:05.191 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:25:05.231 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:25:05.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:25:05.399 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:25:05.418 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:25:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:25:05.473 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:25:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:25:05.546 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:25:05.577 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:25:05.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:25:05.611 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:25:05.614 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:25:05.648 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:25:05.660 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:25:05.695 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:25:05.696 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:25:05.731 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:25:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:25:05.774 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:25:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:25:05.816 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:25:05.918 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:25:05.950 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:25:05.970 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:25:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:25:06.042 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:25:06.069 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:25:06.111 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:25:06.136 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:25:06.201 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:25:06.204 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:25:06.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:25:06.245 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:25:06.266 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:25:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:25:06.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:25:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:25:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:25:06.365 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:25:06.410 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:25:06.492 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:25:06.528 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:25:06.540 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:25:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:25:06.569 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:25:06.597 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:25:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:25:06.631 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:25:06.641 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:25:06.670 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:25:06.673 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:25:06.702 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:25:06.703 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:25:06.729 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:25:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:25:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:25:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:25:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:25:06.804 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:25:06.839 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:25:06.841 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:25:06.876 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:25:06.925 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:25:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:25:06.969 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:25:07.001 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:25:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:25:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:25:07.097 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:25:07.155 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:25:07.210 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:25:07.296 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:25:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:25:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:25:07.925 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:25:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:25:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:25:08.106 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:25:08.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:25:08.203 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:25:08.221 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:25:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:25:08.255 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:25:08.285 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:25:08.353 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:25:08.430 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:25:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:25:08.508 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:25:08.533 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:25:08.541 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:25:08.611 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:25:08.726 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:25:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:25:08.816 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:25:09.250 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:25:09.364 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:25:09.454 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:25:09.563 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:25:10.583 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:25:10.647 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:25:10.674 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:25:10.708 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:25:10.779 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:25:10.856 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:25:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:25:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:25:11.028 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:25:11.071 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:25:11.161 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:25:11.192 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:25:11.228 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:25:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:25:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:25:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:25:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:25:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:25:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:25:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:25:11.836 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:25:11.858 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:25:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:25:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:25:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:25:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:25:14.261 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:25:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:25:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:25:14.455 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:25:14.869 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:25:14.907 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:25:14.996 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:25:14.997 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:25:14.997 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:25:14.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:15.017 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:25:15.026 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:25:15.067 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:25:15.159 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:25:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:25:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:25:15.159 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:15.211 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:25:26.892 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:25:26.892 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-25T16:25:26.892 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-25T16:25:26.892 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-25T16:25:26.892 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-25T16:25:26.893 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-25T16:25:26.893 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-25T16:25:26.893 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-25T16:25:26.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:25:26.978 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-25T16:25:26.978 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-25T16:25:26.978 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-25T16:25:26.978 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-25T16:25:26.979 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-25T16:25:26.979 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-25T16:25:26.979 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-25T16:25:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:27.071 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:25:27.087 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:25:27.087 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:27.087 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:25:27.087 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:25:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:25:27.174 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:27.174 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:25:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:25:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:25:28.667 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:28.667 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:25:28.667 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:25:28.668 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:25:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:28.753 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:25:28.779 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:25:28.779 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:28.779 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:25:28.779 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:25:28.780 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:25:28.780 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:28.892 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:25:28.915 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:25:28.915 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:28.915 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:25:28.915 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:25:28.915 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:25:28.915 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:29.012 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:25:29.034 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:25:29.035 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:29.035 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:25:29.035 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:25:29.035 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:25:29.035 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:29.377 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:25:29.399 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:25:29.399 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:29.399 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:25:29.400 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:25:29.400 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:25:29.400 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:25:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:25:29.518 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:25:29.518 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:29.519 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:25:29.519 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:25:29.519 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:25:29.519 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:29.547 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:25:29.602 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:25:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:25:29.683 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:25:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:25:29.794 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:25:29.854 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:25:29.924 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:25:29.931 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:25:29.980 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:25:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:25:30.066 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:25:30.156 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:25:30.158 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:25:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:25:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:25:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:25:30.286 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:25:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:25:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:25:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:25:30.692 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:25:30.722 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:25:30.754 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:25:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:25:30.824 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:25:30.848 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:25:30.862 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:25:30.880 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:25:30.941 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:25:30.979 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:25:31.893 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:25:31.923 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:25:32.020 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:25:32.056 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:25:32.884 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:25:32.915 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:25:33.015 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:25:33.022 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:25:33.048 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:25:33.149 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:25:33.173 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:25:33.201 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:25:33.224 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:25:33.224 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:33.224 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:25:33.224 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:25:33.224 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:25:33.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:33.269 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:25:33.279 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:25:33.315 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:25:33.350 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:25:33.373 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:25:33.373 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:33.373 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:25:33.373 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:25:33.373 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:25:33.373 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:33.377 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:25:33.418 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:25:33.425 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:25:33.428 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:25:33.528 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:25:33.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:25:34.367 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:25:34.405 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:25:34.428 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:25:34.428 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.428 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:25:34.428 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:25:34.428 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:25:34.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:34.473 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:25:34.494 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:25:34.494 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.494 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:25:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:25:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:25:34.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:25:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:25:34.562 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.562 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:25:34.562 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:25:34.563 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:25:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:34.568 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:25:34.568 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.568 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:25:34.568 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:25:34.568 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:25:34.569 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:34.613 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:25:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:25:34.634 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.634 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:25:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:34.680 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:25:34.702 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:25:34.702 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.702 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:25:34.702 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:25:34.703 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:25:34.703 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:34.874 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:25:34.896 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:25:34.896 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:34.897 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:25:34.897 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:25:34.897 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:25:34.897 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:25:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:25:35.036 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:25:35.037 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:25:35.037 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:25:35.037 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:25:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:39.146 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:25:39.187 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:25:39.228 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:25:39.260 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:25:39.287 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:25:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:25:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:25:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:25:39.433 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:25:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:25:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:25:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:25:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:25:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:25:39.728 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:25:40.123 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:25:40.161 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:25:40.181 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:25:40.298 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:25:40.359 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:25:40.379 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:25:41.002 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:25:41.089 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:25:41.130 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:25:41.209 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:25:41.210 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:25:41.243 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:25:41.243 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:25:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:25:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:25:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:25:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:25:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:25:41.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:25:41.432 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:25:41.460 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:25:41.460 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:25:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:25:41.547 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:25:41.580 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:25:41.580 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:25:41.672 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:25:41.737 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:25:47.615 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:25:47.616 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:25:47.617 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:25:47.618 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:25:47.629 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:25:47.630 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:25:47.631 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:25:47.632 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:25:47.633 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:25:47.634 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:25:47.635 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:25:47.636 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:25:47.637 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:25:47.637 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:25:47.637 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:25:47.637 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:25:47.637 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:25:47.637 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:25:47.638 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:25:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:25:47.919 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:25:47.919 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:25:47.919 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:25:47.919 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:25:47.919 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:25:47.920 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:25:47.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:25:47.932 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:25:47.933 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:25:47.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:25:47.935 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:25:47.936 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:25:47.937 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:25:47.938 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:25:47.939 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.708 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.709 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:25:49.710 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:25:49.711 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:25:49.712 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:25:49.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:25:49.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:25:49.715 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:25:49.716 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.717 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:49.718 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T16:25:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:25:49.985 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.986 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:25:49.987 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:25:49.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:25:49.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:25:49.990 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:25:49.991 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:25:49.992 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:25:49.993 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:25:49.993 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:25:49.993 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:25:49.993 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:49.993 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T16:25:50.157 DEBUG:teuthology.parallel:result is None 2024-04-25T16:25:50.428 DEBUG:teuthology.parallel:result is None 2024-04-25T16:25:50.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:25:50.571 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:25:50.593 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:25:50.594 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:25:50.594 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:25:50.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:25:50.734 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:25:50.755 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:25:50.755 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:25:50.755 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:25:50.757 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T16:25:50.757 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:50.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:25:50.783 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:50.783 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:25:50.822 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T16:25:50.823 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:50.823 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:25:50.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:25:50.912 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:50.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:25:50.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:25:51.001 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T16:25:51.001 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:51.001 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:25:51.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:25:51.090 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:51.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:25:51.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:25:51.178 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T16:25:51.179 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:51.179 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:25:51.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:25:51.265 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:51.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:25:51.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:25:51.353 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T16:25:51.448 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T16:25:51.448 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-04-25T16:25:51.450 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-04-25T16:25:51.474 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T16:25:51.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T16:25:51.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T16:25:51.538 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T16:25:51.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T16:25:51.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T16:25:51.607 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T16:25:51.608 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T16:25:51.608 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T16:25:51.608 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:51.608 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T16:25:51.653 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T16:25:51.762 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T16:25:51.800 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:51.800 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T16:25:51.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T16:25:51.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T16:25:51.965 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T16:25:51.966 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_oc_max_dirty_age': 1, 'debug client': 20, 'debug ms': 1, 'ms_inject_delay_max': 5, 'ms_inject_delay_probability': 1, 'ms_inject_delay_type': 'osd', 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T16:25:51.966 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/truncate_delay}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673262', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client_oc_max_dirty_age': 1, 'debug client': 20, 'debug ms': 1, 'ms_inject_delay_max': 5, 'ms_inject_delay_probability': 1, 'ms_inject_delay_type': 'osd', 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL39k2GKCSqtAi+vOrU1SgEJA4t9kGe6dmfHNeLhnWhpXsNdB4cWycOPWUrAUu2FkJYZPLLgZfQb0z2tPunGzMc=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ+yrv5lpOeHMsjAEWeY73vZ/jZYbu55LKVMkLHX81jDUrmrx5OV+KVQgFDC1uQQD6A8po2iFH20LBJQ2zwEktk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client_oc_max_dirty_age': 1, 'debug client': 20, 'debug ms': 1, 'ms_inject_delay_max': 5, 'ms_inject_delay_probability': 1, 'ms_inject_delay_type': 'osd', 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'exec': {'client.0': ['cd $TESTDIR/mnt.0 && dd if=/dev/zero of=./foo count=100', 'sleep 2', 'cd $TESTDIR/mnt.0 && truncate --size 0 ./foo']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T16:25:51.966 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T16:25:51.969 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T16:25:52.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T16:25:52.024 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T16:25:52.079 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:52.079 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:25:52.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:25:52.100 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:25:41.867032907 +0000 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:23:58.252385108 +0000 2024-04-25T16:25:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:23:58.252385108 +0000 2024-04-25T16:25:52.155 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 16:23:58.252385108 +0000 2024-04-25T16:25:52.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:25:52.215 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:25:52.215 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:25:52.215 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.2625e-05 s, 6.2 MB/s 2024-04-25T16:25:52.216 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:25:52.272 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-25T16:25:52.327 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:25:52.327 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:52.327 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T16:25:52.328 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:52.328 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:52.328 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:25:41.867032907 +0000 2024-04-25T16:25:52.328 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:23:58.571379181 +0000 2024-04-25T16:25:52.328 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:23:58.571379181 +0000 2024-04-25T16:25:52.328 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 16:23:58.571379181 +0000 2024-04-25T16:25:52.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:25:52.390 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:25:52.390 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:25:52.390 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.7116e-05 s, 5.9 MB/s 2024-04-25T16:25:52.391 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:25:52.446 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:25:41.867032907 +0000 2024-04-25T16:25:52.502 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:23:58.891373237 +0000 2024-04-25T16:25:52.503 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:23:58.891373237 +0000 2024-04-25T16:25:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 16:23:58.891373237 +0000 2024-04-25T16:25:52.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:25:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:25:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:25:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.691e-05 s, 5.9 MB/s 2024-04-25T16:25:52.564 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:25:52.620 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-25T16:25:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:25:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:25:41.868032890 +0000 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:23:59.216367199 +0000 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:23:59.216367199 +0000 2024-04-25T16:25:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 16:23:59.216367199 +0000 2024-04-25T16:25:52.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:25:52.736 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:25:52.737 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:25:52.737 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.307e-05 s, 6.2 MB/s 2024-04-25T16:25:52.738 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:25:52.793 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:25:52.793 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:52.793 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:25:52.811 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:25:52.811 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:25:42.136848232 +0000 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:23:58.282158385 +0000 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:23:58.282158385 +0000 2024-04-25T16:25:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 16:23:58.282158385 +0000 2024-04-25T16:25:52.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:25:52.933 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:25:52.933 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:25:52.933 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000280254 s, 1.8 MB/s 2024-04-25T16:25:52.934 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:25:52.991 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-25T16:25:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:25:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:53.048 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T16:25:53.049 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:53.049 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:53.049 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:25:42.136848232 +0000 2024-04-25T16:25:53.049 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:23:58.620151204 +0000 2024-04-25T16:25:53.049 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:23:58.620151204 +0000 2024-04-25T16:25:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 16:23:58.620151204 +0000 2024-04-25T16:25:53.049 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:25:53.113 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:25:53.113 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:25:53.113 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000193614 s, 2.6 MB/s 2024-04-25T16:25:53.114 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:25:53.170 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:25:42.136848232 +0000 2024-04-25T16:25:53.226 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:23:58.963143917 +0000 2024-04-25T16:25:53.227 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:23:58.963143917 +0000 2024-04-25T16:25:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 16:23:58.963143917 +0000 2024-04-25T16:25:53.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:25:53.289 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:25:53.289 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:25:53.289 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000201603 s, 2.5 MB/s 2024-04-25T16:25:53.291 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:25:53.347 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-25T16:25:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:25:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:25:53.403 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T16:25:53.403 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:25:53.404 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:25:53.404 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:25:42.137848212 +0000 2024-04-25T16:25:53.404 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:23:59.297136822 +0000 2024-04-25T16:25:53.404 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:23:59.297136822 +0000 2024-04-25T16:25:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 16:23:59.297136822 +0000 2024-04-25T16:25:53.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:25:53.466 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:25:53.466 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:25:53.466 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00018988 s, 2.7 MB/s 2024-04-25T16:25:53.467 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:25:53.522 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:25:53.522 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T16:25:53.522 INFO:tasks.ceph:Generating config... 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] client_oc_max_dirty_age = 1 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] ms_inject_delay_max = 5 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] ms_inject_delay_probability = 1 2024-04-25T16:25:53.524 INFO:tasks.ceph:[client] ms_inject_delay_type = osd 2024-04-25T16:25:53.525 INFO:tasks.ceph:[client] pid file = 2024-04-25T16:25:53.525 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T16:25:53.525 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T16:25:53.525 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T16:25:53.525 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T16:25:53.526 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T16:25:53.526 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T16:25:53.526 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T16:25:53.526 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T16:25:53.526 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T16:25:53.526 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T16:25:53.527 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T16:25:53.527 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T16:25:53.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T16:25:53.598 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T16:25:53.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T16:25:53.652 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T16:25:53.675 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.94'), ('mon.b', '172.21.15.138'), ('mon.c', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]')] 2024-04-25T16:25:53.675 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.94,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_oc_max_dirty_age': 1, 'debug client': 20, 'debug ms': 1, 'ms_inject_delay_max': 5, 'ms_inject_delay_probability': 1, 'ms_inject_delay_type': 'osd', 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T16:25:53.676 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:53.676 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T16:25:53.730 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.94 --add b 172.21.15.138 --addv c '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:25:53.824 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:25:53.824 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: generated fsid e3c627c0-5913-4b5c-b1aa-d445103b794b 2024-04-25T16:25:53.824 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-04-25T16:25:53.824 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 0 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:fsid e3c627c0-5913-4b5c-b1aa-d445103b794b 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:last_changed 2024-04-25T16:25:53.824611+0000 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-25T16:25:53.824611+0000 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:min_mon_release 17 (quincy) 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:election_strategy: 1 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:2: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.c 2024-04-25T16:25:53.825 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T16:25:53.826 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T16:25:53.842 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e3c627c0-5913-4b5c-b1aa-d445103b794b... 2024-04-25T16:25:53.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T16:25:53.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T16:25:53.922 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-04-25T16:25:53.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-04-25T16:25:53.924 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.94,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-04-25T16:25:53.925 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = e3c627c0-5913-4b5c-b1aa-d445103b794b 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-04-25T16:25:53.926 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.927 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 20 2024-04-25T16:25:53.928 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 20 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 10 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-04-25T16:25:53.929 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-04-25T16:25:53.930 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-04-25T16:25:53.931 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: mon op complaint time = 120 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 600 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: client_oc_max_dirty_age = 1 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:25:53.932 INFO:teuthology.orchestra.run.smithi094.stdout: ms_inject_delay_max = 5 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout: ms_inject_delay_probability = 1 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout: ms_inject_delay_type = osd 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-04-25T16:25:53.933 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds = 20 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds balancer = 20 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug frag = True 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug scatterstat = True 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: mds op complaint time = 180 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: mds verify scatter = True 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2024-04-25T16:25:53.934 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2024-04-25T16:25:53.959 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-04-25T16:25:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-04-25T16:25:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T16:25:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-04-25T16:25:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-04-25T16:25:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-04-25T16:25:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.94,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on skipped message = False 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = e3c627c0-5913-4b5c-b1aa-d445103b794b 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-04-25T16:25:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-04-25T16:25:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-04-25T16:25:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-04-25T16:25:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: client_oc_max_dirty_age = 1 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: ms_inject_delay_max = 5 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: ms_inject_delay_probability = 1 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: ms_inject_delay_type = osd 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-04-25T16:25:53.969 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-04-25T16:25:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-04-25T16:25:53.971 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T16:25:53.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T16:25:54.037 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T16:25:54.038 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:54.038 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T16:25:54.052 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:54.053 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T16:25:54.108 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:25:54.108 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:54.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T16:25:54.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T16:25:54.180 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:25:54.180 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:25:54.234 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:25:54.235 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:54.235 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T16:25:54.235 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T16:25:54.269 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:25:54.269 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:25:54.326 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T16:25:54.326 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T16:25:54.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T16:25:54.384 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T16:25:54.387 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T16:25:54.467 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T16:25:54.470 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T16:25:54.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T16:25:54.528 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T16:25:54.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T16:25:54.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T16:25:54.651 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T16:25:54.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T16:25:54.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T16:25:54.737 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T16:25:54.740 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T16:25:54.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T16:25:54.862 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T16:25:54.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T16:25:54.888 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T16:25:54.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T16:25:54.937 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T16:25:54.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T16:25:54.997 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T16:25:55.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T16:25:55.058 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T16:25:55.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T16:25:55.118 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T16:25:55.130 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T16:25:55.131 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T16:25:55.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T16:25:55.154 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:25:55.154 INFO:tasks.ceph:role: osd.0 2024-04-25T16:25:55.154 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:25:55.154 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:25:55.218 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:25:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:25:55.219 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:25:55.251 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:25:55.252 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:25:55.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T16:25:55.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T16:25:55.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T16:25:55.448 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:25:55.448 INFO:tasks.ceph:role: osd.1 2024-04-25T16:25:55.448 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:25:55.448 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T16:25:55.510 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:25:55.510 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:25:55.511 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:25:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:25:55.545 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:25:55.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T16:25:55.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T16:25:55.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T16:25:55.743 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:25:55.743 INFO:tasks.ceph:role: osd.2 2024-04-25T16:25:55.743 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:25:55.743 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:25:55.805 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:25:55.837 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:25:55.838 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:25:55.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T16:25:55.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T16:25:55.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T16:25:56.037 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:25:56.037 INFO:tasks.ceph:role: osd.3 2024-04-25T16:25:56.037 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:25:56.037 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:25:56.101 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:25:56.134 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:25:56.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:25:56.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T16:25:56.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T16:25:56.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:25:56.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:56.367+0000 7f39fe123640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T16:25:56.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:56.367+0000 7f39fe123640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T16:25:56.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:56.367+0000 7f39fe123640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T16:25:56.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:56.367+0000 7f39fe123640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T16:25:56.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:56.367+0000 7f39fe123640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T16:25:56.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:56.367+0000 7f39fe123640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T16:25:59.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T16:25:59.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:25:59.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:59.348+0000 7fda62a0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T16:25:59.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:59.349+0000 7fda62a0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T16:25:59.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:59.349+0000 7fda62a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T16:25:59.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:59.349+0000 7fda62a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T16:25:59.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:59.349+0000 7fda62a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T16:25:59.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:25:59.349+0000 7fda62a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T16:26:02.195 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T16:26:02.223 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:02.320+0000 7f8f9162c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T16:26:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:02.320+0000 7f8f9162c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T16:26:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:02.320+0000 7f8f9162c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T16:26:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:02.320+0000 7f8f9162c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T16:26:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:02.320+0000 7f8f9162c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T16:26:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:02.321+0000 7f8f9162c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T16:26:05.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T16:26:05.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:05.284+0000 7fd8fd20c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T16:26:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:05.284+0000 7fd8fd20c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T16:26:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:05.284+0000 7fd8fd20c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T16:26:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:05.284+0000 7fd8fd20c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T16:26:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:05.284+0000 7fd8fd20c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T16:26:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:05.284+0000 7fd8fd20c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T16:26:08.123 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T16:26:08.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T16:26:08.177 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:26:08.177 INFO:tasks.ceph:role: osd.4 2024-04-25T16:26:08.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:26:08.178 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:26:08.252 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:26:08.762 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:26:08.765 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:26:08.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T16:26:08.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T16:26:08.926 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T16:26:08.989 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:26:08.989 INFO:tasks.ceph:role: osd.5 2024-04-25T16:26:08.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:26:08.990 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:26:09.061 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:26:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:26:09.062 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:26:09.517 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:26:09.519 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:26:09.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T16:26:09.598 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T16:26:09.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T16:26:09.728 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:26:09.728 INFO:tasks.ceph:role: osd.6 2024-04-25T16:26:09.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:26:09.728 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T16:26:09.798 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:26:09.798 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:26:10.300 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:26:10.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:26:10.302 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T16:26:10.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T16:26:10.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T16:26:10.511 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:26:10.511 INFO:tasks.ceph:role: osd.7 2024-04-25T16:26:10.511 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:26:10.511 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T16:26:10.582 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:26:10.582 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:26:10.582 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:26:10.582 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T16:26:10.583 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:26:10.583 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:26:10.583 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:26:10.583 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T16:26:10.583 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:26:10.583 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:26:11.071 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:26:11.073 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:26:11.073 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T16:26:11.195 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T16:26:11.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:11.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:11.318+0000 7f63464ea640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T16:26:11.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:11.319+0000 7f63464ea640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T16:26:11.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:11.319+0000 7f63464ea640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T16:26:11.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:11.319+0000 7f63464ea640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T16:26:11.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:11.319+0000 7f63464ea640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T16:26:11.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:11.319+0000 7f63464ea640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T16:26:14.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T16:26:14.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:14.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:14.299+0000 7f9d6c123640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T16:26:14.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:14.300+0000 7f9d6c123640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T16:26:14.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:14.300+0000 7f9d6c123640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T16:26:14.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:14.300+0000 7f9d6c123640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T16:26:14.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:14.300+0000 7f9d6c123640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T16:26:14.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:14.300+0000 7f9d6c123640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T16:26:17.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T16:26:17.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:17.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:17.265+0000 7f667c29d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T16:26:17.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:17.265+0000 7f667c29d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T16:26:17.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:17.265+0000 7f667c29d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T16:26:17.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:17.265+0000 7f667c29d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T16:26:17.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:17.265+0000 7f667c29d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T16:26:17.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:17.266+0000 7f667c29d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T16:26:20.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T16:26:20.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:20.237+0000 7f74be218640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T16:26:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:20.238+0000 7f74be218640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T16:26:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:20.238+0000 7f74be218640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T16:26:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:20.238+0000 7f74be218640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T16:26:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:20.238+0000 7f74be218640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T16:26:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:20.238+0000 7f74be218640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T16:26:21.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:26:21.970 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:26:23.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T16:26:23.121 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T16:26:23.121 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T16:26:23.150 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T16:26:23.219 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.219 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T16:26:23.285 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T16:26:23.352 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T16:26:23.417 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T16:26:23.483 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T16:26:23.548 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T16:26:23.576 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T16:26:23.643 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.643 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T16:26:23.709 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T16:26:23.775 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.775 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T16:26:23.840 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T16:26:23.904 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:23.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T16:26:23.969 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.970 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T16:26:23.987 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:23.987 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T16:26:24.044 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:24.045 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T16:26:24.060 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:24.060 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T16:26:24.114 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T16:26:24.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T16:26:24.117 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T16:26:24.158 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-04-25T16:26:24.158 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpm+R3wFhAAYKI0oqS9/k765eqAhx+3uA== 2024-04-25T16:26:24.158 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-04-25T16:26:24.158 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpmTlGAHxAAYax/FJX+BewLBRKEs8Jtig== 2024-04-25T16:26:24.158 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpm7qbTJhAAfuLDdWw6PxostiONpJvTTQ== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAUhCpmDGT3FRAAwevGLNIR0m7qcBzVUNNX5Q== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAXhCpmznncFBAArO/hwFZG9LX0f8Ez5FtSRw== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAahCpmVNQuExAAe9G4K6U5t7nSoDY8E6PhTg== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAdhCpmUecBERAAtEkygxrQrD5IhvI2EHJFmw== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpmhsHfGxAAjDBehbWa/age81TyRg7Rfg== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpmu4X4KxAAZFiKTCr4ZvvvhpmD2qswPg== 2024-04-25T16:26:24.159 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpmLilsMxAAf4o5DSW+tK4a63BpUmi3Mw== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAjhCpm4GEUExAAnr5YJNiYEK9vsnrriWr3yw== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAmhCpmdkXiERAAITooD0ZToHmvmwP+0JQszw== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAphCpmETfSDxAAGPEREafzp/cWswuS2su+wQ== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAshCpmonYzDhAAv1BDVXohJVOWGET2blEnQQ== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpm0U/hNxAAcL/Ap80tCGCl+txVlG9Fjg== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAShCpmX192OxAAwKQ0mYPA7uOa9P93BNaVHg== 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-04-25T16:26:24.160 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAThCpmnpF8AxAA0v/cwikUsANrdgiKCosJqg== 2024-04-25T16:26:24.161 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2024-04-25T16:26:24.161 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAThCpm64gRBxAAvyaHNO218ejA9zbhxPd7ZA== 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpm+R3wFhAAYKI0oqS9/k765eqAhx+3uA== 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpmTlGAHxAAYax/FJX+BewLBRKEs8Jtig== 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpm7qbTJhAAfuLDdWw6PxostiONpJvTTQ== 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-04-25T16:26:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAUhCpmDGT3FRAAwevGLNIR0m7qcBzVUNNX5Q== 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAXhCpmznncFBAArO/hwFZG9LX0f8Ez5FtSRw== 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAahCpmVNQuExAAe9G4K6U5t7nSoDY8E6PhTg== 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAdhCpmUecBERAAtEkygxrQrD5IhvI2EHJFmw== 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpmhsHfGxAAjDBehbWa/age81TyRg7Rfg== 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpmu4X4KxAAZFiKTCr4ZvvvhpmD2qswPg== 2024-04-25T16:26:24.178 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpmLilsMxAAf4o5DSW+tK4a63BpUmi3Mw== 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAjhCpm4GEUExAAnr5YJNiYEK9vsnrriWr3yw== 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAmhCpmdkXiERAAITooD0ZToHmvmwP+0JQszw== 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAphCpmETfSDxAAGPEREafzp/cWswuS2su+wQ== 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAshCpmonYzDhAAv1BDVXohJVOWGET2blEnQQ== 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-04-25T16:26:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpm0U/hNxAAcL/Ap80tCGCl+txVlG9Fjg== 2024-04-25T16:26:24.180 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2024-04-25T16:26:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAShCpmX192OxAAwKQ0mYPA7uOa9P93BNaVHg== 2024-04-25T16:26:24.180 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-04-25T16:26:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAThCpmnpF8AxAA0v/cwikUsANrdgiKCosJqg== 2024-04-25T16:26:24.180 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2024-04-25T16:26:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAThCpm64gRBxAAvyaHNO218ejA9zbhxPd7ZA== 2024-04-25T16:26:24.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:26:24.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:26:24.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.410 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.592 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:26:24.653 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:26:24.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:26:24.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.900 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.958 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:24.960 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:25.019 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:25.021 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:26:25.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.206 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:26:25.333 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T16:26:25.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T16:26:25.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T16:26:25.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T16:26:25.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T16:26:25.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T16:26:25.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T16:26:25.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T16:26:25.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T16:26:25.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T16:26:25.828 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:25.830 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:26:25.883 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T16:26:25.883 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T16:26:25.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T16:26:25.886 INFO:tasks.ceph.mon.a:Started 2024-04-25T16:26:25.886 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T16:26:25.886 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T16:26:25.925 INFO:tasks.ceph.mon.b:Started 2024-04-25T16:26:25.925 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T16:26:25.926 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T16:26:25.928 INFO:tasks.ceph.mon.c:Started 2024-04-25T16:26:25.928 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T16:26:25.928 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T16:26:25.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T16:26:25.931 INFO:tasks.ceph.mgr.y:Started 2024-04-25T16:26:25.931 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T16:26:25.931 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T16:26:25.933 INFO:tasks.ceph.mgr.x:Started 2024-04-25T16:26:25.933 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T16:26:25.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T16:26:26.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.042+0000 7f984b3a8640 1 Processor -- start 2024-04-25T16:26:26.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.042+0000 7f984b3a8640 1 -- start start 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f984b3a8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f984b3a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f984b3a8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f984406c980 con 0x7f984415d4b0 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f984b3a8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f984415d8c0 con 0x7f984406c570 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f984b3a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f984406e580 con 0x7f984406dc30 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9848943640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9849144640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f984415d4b0 0x7f98440b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56364/0 (socket says 172.21.15.94:56364) 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 learned_addr learned my addr 172.21.15.94:0/73842678 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:26.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.043+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:26:26.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.044+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:26.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.044+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T16:26:26.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.044+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:26.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.044+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:26.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.044+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:26:26.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:26.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.245+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:26:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T16:26:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:26.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:26:26.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:26.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:26:26.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:26.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:26.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:26.646+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:26:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9848943640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:27.447+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 msgr2=0x7f984406e040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f984406dc30 0x7f984406e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 msgr2=0x7f984415e4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.94:3300/0 conn(0x7f984406c570 0x7f984415e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 -- 172.21.15.94:0/73842678 >> v1:172.21.15.94:6789/0 conn(0x7f984415d4b0 legacy=0x7f98440b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 -- 172.21.15.94:0/73842678 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f984406cc10 con 0x7f98280061f0 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f983affd640 1 -- 172.21.15.94:0/73842678 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f984415d8c0 con 0x7f98280036f0 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:29.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:26:29.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:29.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:29.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.044+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:26:29.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.245+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:29.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.245+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:26:29.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.245+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:29.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.245+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:29.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.245+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:26:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.646+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.646+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:26:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.646+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.646+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:29.646+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:26:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:30.448+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:30.448+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:26:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:30.448+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:26:30.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:30.448+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:26:30.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:30.448+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:26:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.212+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1119828750 0 0) 0x7f984406cc10 con 0x7f98280061f0 2024-04-25T16:26:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.212+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9830005730 con 0x7f98280061f0 2024-04-25T16:26:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2805988226 0 0) 0x7f9830005730 con 0x7f98280061f0 2024-04-25T16:26:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 msgr2=0x7f9828005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:26:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 --2- 172.21.15.94:0/73842678 >> v2:172.21.15.138:3300/0 conn(0x7f98280036f0 0x7f9828005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982800af80 con 0x7f98280061f0 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f983400d580 con 0x7f98280061f0 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 >> v1:172.21.15.138:6789/0 conn(0x7f98280061f0 legacy=0x7f98280086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f9849945640 1 -- 172.21.15.94:0/73842678 reap_dead start 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9830006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830006a10 0x7f9830006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830007590 0x7f983000dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f984415d8c0 con 0x7f9830005bc0 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9830006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f983000e530 con 0x7f9830007590 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.213+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f983000e730 con 0x7f9830006a10 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9849945640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830007590 0x7f983000dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9848943640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830006a10 0x7f9830006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830006a10 msgr2=0x7f9830006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830006a10 0x7f9830006e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830007590 msgr2=0x7f983000dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830007590 0x7f983000dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.214+0000 7f9849144640 1 -- 172.21.15.94:0/73842678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9830005730 con 0x7f9830005bc0 2024-04-25T16:26:31.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.215+0000 7f9849144640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9830006010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9840007d20 tx=0x7f9830007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ebaa9b6e6e422e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.217+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840031070 con 0x7f9830005bc0 2024-04-25T16:26:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.217+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984002b810 con 0x7f9830005bc0 2024-04-25T16:26:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.217+0000 7f983bfff640 1 -- 172.21.15.94:0/73842678 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840005bc0 con 0x7f9830005bc0 2024-04-25T16:26:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 -- 172.21.15.94:0/73842678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 msgr2=0x7f9830006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9830006010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9840007d20 tx=0x7f9830007a40 comp rx=0 tx=0).stop 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 -- 172.21.15.94:0/73842678 shutdown_connections 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830007590 0x7f983000dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830006a10 0x7f9830006e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 --2- 172.21.15.94:0/73842678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9830006010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 -- 172.21.15.94:0/73842678 >> 172.21.15.94:0/73842678 conn(0x7f984409b9b0 msgr2=0x7f98440b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 -- 172.21.15.94:0/73842678 shutdown_connections 2024-04-25T16:26:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.218+0000 7f984b3a8640 1 -- 172.21.15.94:0/73842678 wait complete. 2024-04-25T16:26:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.221+0000 7f984b3a8640 1 Processor -- start 2024-04-25T16:26:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.221+0000 7f984b3a8640 1 -- start start 2024-04-25T16:26:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f984b3a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9844158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f984b3a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830006a10 0x7f984415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f984b3a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830007590 0x7f984415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f984b3a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f984406d430 con 0x7f9830005bc0 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f984b3a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f984406ceb0 con 0x7f9830006a10 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f984b3a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f984406d710 con 0x7f9830007590 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f9849144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9844158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f9849144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9844158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33026/0 (socket says 172.21.15.94:33026) 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f9848943640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830006a10 0x7f984415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f9849144640 1 -- 172.21.15.94:0/1554579488 learned_addr learned my addr 172.21.15.94:0/1554579488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.222+0000 7f9849945640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830007590 0x7f984415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9848943640 1 -- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830007590 msgr2=0x7f984415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9848943640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830007590 0x7f984415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9848943640 1 -- 172.21.15.94:0/1554579488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 msgr2=0x7f9844158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9848943640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9844158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9848943640 1 -- 172.21.15.94:0/1554579488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98440cbd90 con 0x7f9830006a10 2024-04-25T16:26:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9848943640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830006a10 0x7f984415a3d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98440b8290 tx=0x7f9834016e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834014c30 con 0x7f9830006a10 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983401b070 con 0x7f9830006a10 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.223+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98440cee00 con 0x7f9830006a10 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.224+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834020470 con 0x7f9830006a10 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.224+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984415efb0 con 0x7f9830006a10 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.225+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98340209c0 con 0x7f9830006a10 2024-04-25T16:26:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.225+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9834029730 con 0x7f9830006a10 2024-04-25T16:26:31.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.225+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984406a180 con 0x7f9830006a10 2024-04-25T16:26:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.228+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f983402e020 con 0x7f9830006a10 2024-04-25T16:26:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.296+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f98440b3f00 con 0x7f9830006a10 2024-04-25T16:26:31.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.300+0000 7f9839ffb640 1 -- 172.21.15.94:0/1554579488 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9834017df0 con 0x7f9830006a10 2024-04-25T16:26:31.300 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830006a10 msgr2=0x7f984415a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830006a10 0x7f984415a3d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98440b8290 tx=0x7f9834016e80 comp rx=0 tx=0).stop 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 shutdown_connections 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9830007590 0x7f984415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9830006a10 0x7f984415a3d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 --2- 172.21.15.94:0/1554579488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830005bc0 0x7f9844158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 >> 172.21.15.94:0/1554579488 conn(0x7f984409b9b0 msgr2=0x7f98440b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 shutdown_connections 2024-04-25T16:26:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:31.301+0000 7f984b3a8640 1 -- 172.21.15.94:0/1554579488 wait complete. 2024-04-25T16:26:31.314 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T16:26:31.314 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T16:26:31.314 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:31.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T16:26:31.334 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:31.333+0000 7f61a583d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:26:31.338 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:31.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T16:26:31.345 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:31.344+0000 7f0945796040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:26:31.402 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:31.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T16:26:31.465 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:26:31.466 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T16:26:31.485 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:31.484+0000 7f61a583d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:26:31.495 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:31.494+0000 7f0945796040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:26:31.508 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:31.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T16:26:31.531 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:31.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T16:26:31.541 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:31.540+0000 7f61a583d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:26:31.551 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:31.550+0000 7f0945796040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:26:31.573 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:31.573 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T16:26:31.637 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:26:31.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T16:26:31.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new dfe00173-43d3-4c80-aafc-55f29fe7e2b9 0 2024-04-25T16:26:31.787 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:31.787+0000 7f61a583d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:26:31.795 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:31.794+0000 7f0945796040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:26:31.832 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:31.831+0000 7f61a583d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:26:31.844 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:31.843+0000 7f0945796040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:26:31.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.848+0000 7febddcaa640 1 Processor -- start 2024-04-25T16:26:31.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.848+0000 7febddcaa640 1 -- start start 2024-04-25T16:26:31.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febddcaa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febd806f130 0x7febd809c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febddcaa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7febd80ce470 0x7febd80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febddcaa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd806f540 con 0x7febd806df10 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febddcaa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7febd806f7c0 con 0x7febd80ce470 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febddcaa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7febd8070670 con 0x7febd806f130 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febd806f130 0x7febd809c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febd806f130 0x7febd809c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46310/0 (socket says 172.21.15.138:46310) 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 -- 172.21.15.138:0/2645825551 learned_addr learned my addr 172.21.15.138:0/2645825551 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd7fff640 1 --2- 172.21.15.138:0/2645825551 >> v2:172.21.15.94:3300/0 conn(0x7febd80ce470 0x7febd80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 -- 172.21.15.138:0/2645825551 >> v2:172.21.15.94:3300/0 conn(0x7febd80ce470 msgr2=0x7febd80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 --2- 172.21.15.138:0/2645825551 >> v2:172.21.15.94:3300/0 conn(0x7febd80ce470 0x7febd80ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 -- 172.21.15.138:0/2645825551 >> v1:172.21.15.138:6789/0 conn(0x7febd806df10 legacy=0x7febd806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.849+0000 7febd77fe640 1 -- 172.21.15.138:0/2645825551 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8070950 con 0x7febd806f130 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.850+0000 7febd77fe640 1 --2- 172.21.15.138:0/2645825551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809c530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febd80d3f10 tx=0x7febcc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e724ffa8c11f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.850+0000 7febd67fc640 1 -- 172.21.15.138:0/2645825551 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc037070 con 0x7febd806f130 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.850+0000 7febd67fc640 1 -- 172.21.15.138:0/2645825551 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc032440 con 0x7febd806f130 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.850+0000 7febd67fc640 1 -- 172.21.15.138:0/2645825551 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc032810 con 0x7febd806f130 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 -- 172.21.15.138:0/2645825551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 msgr2=0x7febd809c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 --2- 172.21.15.138:0/2645825551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809c530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febd80d3f10 tx=0x7febcc02a270 comp rx=0 tx=0).stop 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 -- 172.21.15.138:0/2645825551 shutdown_connections 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 --2- 172.21.15.138:0/2645825551 >> v2:172.21.15.94:3300/0 conn(0x7febd80ce470 0x7febd80ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 --2- 172.21.15.138:0/2645825551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809c530 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 -- 172.21.15.138:0/2645825551 >> 172.21.15.138:0/2645825551 conn(0x7febd80b7b60 msgr2=0x7febd80b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 -- 172.21.15.138:0/2645825551 shutdown_connections 2024-04-25T16:26:31.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.851+0000 7febddcaa640 1 -- 172.21.15.138:0/2645825551 wait complete. 2024-04-25T16:26:31.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.853+0000 7febddcaa640 1 Processor -- start 2024-04-25T16:26:31.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.853+0000 7febddcaa640 1 -- start start 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.854+0000 7febddcaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd806df10 0x7febd80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.854+0000 7febddcaa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.854+0000 7febddcaa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 0x7febd8151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.854+0000 7febddcaa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febd815d7e0 con 0x7febd806df10 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.854+0000 7febddcaa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febd8070c30 con 0x7febd806f130 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.854+0000 7febddcaa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7febd809dd30 con 0x7febd80ce470 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 0x7febd8151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 0x7febd8151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42586/0 (socket says 172.21.15.138:42586) 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd7fff640 1 -- 172.21.15.138:0/910523778 learned_addr learned my addr 172.21.15.138:0/910523778 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd77fe640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd806df10 0x7febd80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 -- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 msgr2=0x7febd8151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 0x7febd8151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 -- 172.21.15.138:0/910523778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd806df10 msgr2=0x7febd80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd806df10 0x7febd80cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 -- 172.21.15.138:0/910523778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd81f4420 con 0x7febd806f130 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd7fff640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 0x7febd8151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd77fe640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd806df10 0x7febd80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febd6ffd640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809d790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7febc800c3f0 tx=0x7febc800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80052d0 con 0x7febd806f130 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc8005490 con 0x7febd806f130 2024-04-25T16:26:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd80cb7f0 con 0x7febd806f130 2024-04-25T16:26:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8007260 con 0x7febd806f130 2024-04-25T16:26:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd809e260 con 0x7febd806f130 2024-04-25T16:26:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febc8007400 con 0x7febd806f130 2024-04-25T16:26:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.855+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7febc8003b20 con 0x7febd806f130 2024-04-25T16:26:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.856+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd806a180 con 0x7febd806f130 2024-04-25T16:26:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.859+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7febd806f540 con 0x7febd806f130 2024-04-25T16:26:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.930+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dfe00173-43d3-4c80-aafc-55f29fe7e2b9", "id": 0} v 0) -- 0x7febd806c890 con 0x7febd806f130 2024-04-25T16:26:31.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.934+0000 7febbffff640 1 -- 172.21.15.138:0/910523778 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dfe00173-43d3-4c80-aafc-55f29fe7e2b9", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7febc8003d20 con 0x7febd806f130 2024-04-25T16:26:31.935 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 msgr2=0x7febd809d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809d790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7febc800c3f0 tx=0x7febc800c8b0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 shutdown_connections 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febd80ce470 0x7febd8151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febd806f130 0x7febd809d790 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 --2- 172.21.15.138:0/910523778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd806df10 0x7febd80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 >> 172.21.15.138:0/910523778 conn(0x7febd80b7b60 msgr2=0x7febd80ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 shutdown_connections 2024-04-25T16:26:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:31.935+0000 7febddcaa640 1 -- 172.21.15.138:0/910523778 wait complete. 2024-04-25T16:26:31.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new e1d86461-a4cb-42a0-a3c1-9772fdbdf564 1 2024-04-25T16:26:32.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.052+0000 7fa5e4e9f640 1 Processor -- start 2024-04-25T16:26:32.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.052+0000 7fa5e4e9f640 1 -- start start 2024-04-25T16:26:32.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5e4e9f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa5e015e8c0 0x7fa5e00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5e4e9f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5e00b2950 0x7fa5e00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5e4e9f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e00d2e70 con 0x7fa5e00d2a60 2024-04-25T16:26:32.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5e4e9f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e00b3f70 con 0x7fa5e015e8c0 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5e4e9f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e00b3ae0 con 0x7fa5e00b2950 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5d7fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5e00b2950 0x7fa5e00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5d7fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5e00b2950 0x7fa5e00d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46336/0 (socket says 172.21.15.138:46336) 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5d7fff640 1 -- 172.21.15.138:0/976402605 learned_addr learned my addr 172.21.15.138:0/976402605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.054+0000 7fa5de59c640 1 --2- 172.21.15.138:0/976402605 >> v2:172.21.15.94:3300/0 conn(0x7fa5e015e8c0 0x7fa5e00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5d7fff640 1 -- 172.21.15.138:0/976402605 >> v2:172.21.15.94:3300/0 conn(0x7fa5e015e8c0 msgr2=0x7fa5e00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5d7fff640 1 --2- 172.21.15.138:0/976402605 >> v2:172.21.15.94:3300/0 conn(0x7fa5e015e8c0 0x7fa5e00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5d7fff640 1 -- 172.21.15.138:0/976402605 >> v1:172.21.15.138:6789/0 conn(0x7fa5e00d2a60 legacy=0x7fa5e00cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5d7fff640 1 -- 172.21.15.138:0/976402605 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e00b2ed0 con 0x7fa5e00b2950 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5d7fff640 1 --2- 172.21.15.138:0/976402605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00b2950 0x7fa5e00d2520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5e009bee0 tx=0x7fa5c802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daef79c3548bae13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5ddd9b640 1 -- 172.21.15.138:0/976402605 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8037070 con 0x7fa5e00b2950 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.055+0000 7fa5ddd9b640 1 -- 172.21.15.138:0/976402605 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c8032440 con 0x7fa5e00b2950 2024-04-25T16:26:32.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.056+0000 7fa5ddd9b640 1 -- 172.21.15.138:0/976402605 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8032810 con 0x7fa5e00b2950 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.056+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/976402605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00b2950 msgr2=0x7fa5e00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.056+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/976402605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00b2950 0x7fa5e00d2520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5e009bee0 tx=0x7fa5c802a270 comp rx=0 tx=0).stop 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.057+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/976402605 shutdown_connections 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.057+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/976402605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00b2950 0x7fa5e00d2520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.057+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/976402605 >> v2:172.21.15.94:3300/0 conn(0x7fa5e015e8c0 0x7fa5e00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.057+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/976402605 >> 172.21.15.138:0/976402605 conn(0x7fa5e009b9b0 msgr2=0x7fa5e00b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.057+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/976402605 shutdown_connections 2024-04-25T16:26:32.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.057+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/976402605 wait complete. 2024-04-25T16:26:32.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.059+0000 7fa5e4e9f640 1 Processor -- start 2024-04-25T16:26:32.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.059+0000 7fa5e4e9f640 1 -- start start 2024-04-25T16:26:32.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5e4e9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5e00b2950 0x7fa5e0153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5e4e9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00d2a60 0x7fa5e0160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5e4e9f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 0x7fa5e0161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5e4e9f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e00b3770 con 0x7fa5e00b2950 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5e4e9f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e00b3490 con 0x7fa5e00d2a60 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5e4e9f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e00cecc0 con 0x7fa5e015e8c0 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00d2a60 0x7fa5e0160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 0x7fa5e0161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 0x7fa5e0161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42588/0 (socket says 172.21.15.138:42588) 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 -- 172.21.15.138:0/1585969987 learned_addr learned my addr 172.21.15.138:0/1585969987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5de59c640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5e00b2950 0x7fa5e0153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 -- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00d2a60 msgr2=0x7fa5e0160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00d2a60 0x7fa5e0160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 -- 172.21.15.138:0/1585969987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5e00b2950 msgr2=0x7fa5e0153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5e00b2950 0x7fa5e0153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 -- 172.21.15.138:0/1585969987 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e0157140 con 0x7fa5e015e8c0 2024-04-25T16:26:32.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5de59c640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5e00b2950 0x7fa5e0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5d7fff640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00d2a60 0x7fa5e0160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.060+0000 7fa5ded9d640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 0x7fa5e0161360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c760 tx=0x7fa5d800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8003b70 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d8003d10 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800aa90 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e0157690 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e0160620 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa5d800ac30 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.061+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa5d8012510 con 0x7fa5e015e8c0 2024-04-25T16:26:32.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.062+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e006a180 con 0x7fa5e015e8c0 2024-04-25T16:26:32.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.066+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa5e00d2e70 con 0x7fa5e015e8c0 2024-04-25T16:26:32.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.136+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e1d86461-a4cb-42a0-a3c1-9772fdbdf564", "id": 1} v 0) -- 0x7fa5e00cf160 con 0x7fa5e015e8c0 2024-04-25T16:26:32.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5d6ffd640 1 -- 172.21.15.138:0/1585969987 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1d86461-a4cb-42a0-a3c1-9772fdbdf564", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa5d80050d0 con 0x7fa5e015e8c0 2024-04-25T16:26:32.141 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 msgr2=0x7fa5e0161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 0x7fa5e0161360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c760 tx=0x7fa5d800cc20 comp rx=0 tx=0).stop 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 shutdown_connections 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5e015e8c0 0x7fa5e0161360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e00d2a60 0x7fa5e0160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 --2- 172.21.15.138:0/1585969987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5e00b2950 0x7fa5e0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.140+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 >> 172.21.15.138:0/1585969987 conn(0x7fa5e009b9b0 msgr2=0x7fa5e006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.141+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 shutdown_connections 2024-04-25T16:26:32.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.141+0000 7fa5e4e9f640 1 -- 172.21.15.138:0/1585969987 wait complete. 2024-04-25T16:26:32.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new a76ea96d-c603-4215-b852-86446453a829 2 2024-04-25T16:26:32.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.261+0000 7f56d3135640 1 Processor -- start 2024-04-25T16:26:32.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.261+0000 7f56d3135640 1 -- start start 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d3135640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc0705d0 0x7f56cc0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d3135640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56cc09bc20 0x7f56cc09c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d3135640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f56cc0cf810 0x7f56cc0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d3135640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f56cc0d1ef0 con 0x7f56cc0cf810 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d3135640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f56cc160d00 con 0x7f56cc09bc20 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d3135640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc09c570 con 0x7f56cc0705d0 2024-04-25T16:26:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56cbfff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56cc09bc20 0x7f56cc09c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d0ed1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc0705d0 0x7f56cc0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56cbfff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56cc09bc20 0x7f56cc09c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46348/0 (socket says 172.21.15.138:46348) 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56cbfff640 1 -- 172.21.15.138:0/1221955026 learned_addr learned my addr 172.21.15.138:0/1221955026 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56d16d2640 1 --2- 172.21.15.138:0/1221955026 >> v2:172.21.15.94:3300/0 conn(0x7f56cc0cf810 0x7f56cc0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.262+0000 7f56cbfff640 1 -- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc0705d0 msgr2=0x7f56cc0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cbfff640 1 --2- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc0705d0 0x7f56cc0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cbfff640 1 -- 172.21.15.138:0/1221955026 >> v2:172.21.15.94:3300/0 conn(0x7f56cc0cf810 msgr2=0x7f56cc0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cbfff640 1 --2- 172.21.15.138:0/1221955026 >> v2:172.21.15.94:3300/0 conn(0x7f56cc0cf810 0x7f56cc0cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cbfff640 1 -- 172.21.15.138:0/1221955026 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc070ab0 con 0x7f56cc09bc20 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cbfff640 1 --2- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc09bc20 0x7f56cc09c030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0d09a0 tx=0x7f56b802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc258a56b8a7a265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cb7fe640 1 -- 172.21.15.138:0/1221955026 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8037070 con 0x7f56cc09bc20 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cb7fe640 1 -- 172.21.15.138:0/1221955026 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b8032440 con 0x7f56cc09bc20 2024-04-25T16:26:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.263+0000 7f56cb7fe640 1 -- 172.21.15.138:0/1221955026 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8032810 con 0x7f56cc09bc20 2024-04-25T16:26:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 -- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc09bc20 msgr2=0x7f56cc09c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 --2- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc09bc20 0x7f56cc09c030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0d09a0 tx=0x7f56b802a1e0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 -- 172.21.15.138:0/1221955026 shutdown_connections 2024-04-25T16:26:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 --2- 172.21.15.138:0/1221955026 >> v2:172.21.15.94:3300/0 conn(0x7f56cc0cf810 0x7f56cc0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 --2- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc09bc20 0x7f56cc09c030 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 --2- 172.21.15.138:0/1221955026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc0705d0 0x7f56cc0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 -- 172.21.15.138:0/1221955026 >> 172.21.15.138:0/1221955026 conn(0x7f56cc0b7b60 msgr2=0x7f56cc0b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 -- 172.21.15.138:0/1221955026 shutdown_connections 2024-04-25T16:26:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.264+0000 7f56d3135640 1 -- 172.21.15.138:0/1221955026 wait complete. 2024-04-25T16:26:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.266+0000 7f56d3135640 1 Processor -- start 2024-04-25T16:26:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.266+0000 7f56d3135640 1 -- start start 2024-04-25T16:26:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.267+0000 7f56d3135640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 0x7f56cc0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.267+0000 7f56d3135640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc09bc20 0x7f56cc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d3135640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0cf810 0x7f56cc0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d3135640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc071350 con 0x7f56cc0cf810 2024-04-25T16:26:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d3135640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc071070 con 0x7f56cc0705d0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d3135640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc0cf040 con 0x7f56cc09bc20 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 0x7f56cc0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 0x7f56cc0b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46356/0 (socket says 172.21.15.138:46356) 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 -- 172.21.15.138:0/3473253209 learned_addr learned my addr 172.21.15.138:0/3473253209 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc09bc20 0x7f56cc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d16d2640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0cf810 0x7f56cc0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 -- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc09bc20 msgr2=0x7f56cc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc09bc20 0x7f56cc0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 -- 172.21.15.138:0/3473253209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0cf810 msgr2=0x7f56cc0b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0cf810 0x7f56cc0b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 -- 172.21.15.138:0/3473253209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc1529c0 con 0x7f56cc0705d0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d0ed1640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 0x7f56cc0b0810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e870 tx=0x7f56c000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0017070 con 0x7f56cc0705d0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c000ee60 con 0x7f56cc0705d0 2024-04-25T16:26:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc152ca0 con 0x7f56cc0705d0 2024-04-25T16:26:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c000cb80 con 0x7f56cc0705d0 2024-04-25T16:26:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.268+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc153260 con 0x7f56cc0705d0 2024-04-25T16:26:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.269+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56c001a070 con 0x7f56cc0705d0 2024-04-25T16:26:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.269+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f56c001c870 con 0x7f56cc0705d0 2024-04-25T16:26:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.270+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc06a180 con 0x7f56cc0705d0 2024-04-25T16:26:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.272+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f56c000cd20 con 0x7f56cc0705d0 2024-04-25T16:26:32.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.344+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a76ea96d-c603-4215-b852-86446453a829", "id": 2} v 0) -- 0x7f56cc0d0670 con 0x7f56cc0705d0 2024-04-25T16:26:32.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.347+0000 7f56c97fa640 1 -- 172.21.15.138:0/3473253209 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a76ea96d-c603-4215-b852-86446453a829", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f56c001c520 con 0x7f56cc0705d0 2024-04-25T16:26:32.349 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 msgr2=0x7f56cc0b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 0x7f56cc0b0810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e870 tx=0x7f56c000ed30 comp rx=0 tx=0).stop 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 shutdown_connections 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0cf810 0x7f56cc0b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc09bc20 0x7f56cc0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 --2- 172.21.15.138:0/3473253209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0705d0 0x7f56cc0b0810 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 >> 172.21.15.138:0/3473253209 conn(0x7f56cc0b7b60 msgr2=0x7f56cc0b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 shutdown_connections 2024-04-25T16:26:32.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.348+0000 7f56d3135640 1 -- 172.21.15.138:0/3473253209 wait complete. 2024-04-25T16:26:32.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 4b9e445a-59c0-4623-b982-7c8f29b19a53 3 2024-04-25T16:26:32.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.467+0000 7efe56070640 1 Processor -- start 2024-04-25T16:26:32.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.467+0000 7efe56070640 1 -- start start 2024-04-25T16:26:32.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe56070640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efe500ce7c0 0x7efe5015e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe56070640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe500ced80 con 0x7efe5009c330 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe56070640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe500cf060 con 0x7efe500cfe20 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe56070640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7efe500cf990 con 0x7efe500ce7c0 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4ffff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7efe500cfe20 0x7efe500d2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44466/0 (socket says 172.21.15.138:44466) 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4ffff640 1 -- 172.21.15.138:0/681215527 learned_addr learned my addr 172.21.15.138:0/681215527 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4f7fe640 1 --2- 172.21.15.138:0/681215527 >> v2:172.21.15.94:3300/0 conn(0x7efe500ce7c0 0x7efe5015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483316140 0 0) 0x7efe500cf060 con 0x7efe500cfe20 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe38003520 con 0x7efe500cfe20 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964673602 0 0) 0x7efe500ced80 con 0x7efe5009c330 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe500cf060 con 0x7efe5009c330 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4f7fe640 1 -- 172.21.15.138:0/681215527 >> v1:172.21.15.138:6789/0 conn(0x7efe500cfe20 legacy=0x7efe500d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4f7fe640 1 -- 172.21.15.138:0/681215527 >> v1:172.21.15.94:6789/0 conn(0x7efe5009c330 legacy=0x7efe5006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.469+0000 7efe4f7fe640 1 -- 172.21.15.138:0/681215527 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe500cf340 con 0x7efe500ce7c0 2024-04-25T16:26:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.470+0000 7efe4f7fe640 1 --2- 172.21.15.138:0/681215527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe500ce7c0 0x7efe5015e630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe440097a0 tx=0x7efe4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77498f2b1f882608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.470+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44037070 con 0x7efe500ce7c0 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.470+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe44032440 con 0x7efe500ce7c0 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.470+0000 7efe4e7fc640 1 -- 172.21.15.138:0/681215527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44032810 con 0x7efe500ce7c0 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 -- 172.21.15.138:0/681215527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe500ce7c0 msgr2=0x7efe5015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 --2- 172.21.15.138:0/681215527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe500ce7c0 0x7efe5015e630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe440097a0 tx=0x7efe4402a230 comp rx=0 tx=0).stop 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 -- 172.21.15.138:0/681215527 shutdown_connections 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 --2- 172.21.15.138:0/681215527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe500ce7c0 0x7efe5015e630 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 -- 172.21.15.138:0/681215527 >> 172.21.15.138:0/681215527 conn(0x7efe500b7b60 msgr2=0x7efe500b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 -- 172.21.15.138:0/681215527 shutdown_connections 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.471+0000 7efe56070640 1 -- 172.21.15.138:0/681215527 wait complete. 2024-04-25T16:26:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.473+0000 7efe56070640 1 Processor -- start 2024-04-25T16:26:32.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.473+0000 7efe56070640 1 -- start start 2024-04-25T16:26:32.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe56070640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5009c330 0x7efe50150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe56070640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe500ce7c0 0x7efe50160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe56070640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe500cfe20 0x7efe500d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe56070640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe5006d510 con 0x7efe5009c330 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe56070640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe5009c740 con 0x7efe500cfe20 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe56070640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe500c7d10 con 0x7efe500ce7c0 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe4ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe500cfe20 0x7efe500d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe4effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe500ce7c0 0x7efe50160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.474+0000 7efe4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5009c330 0x7efe50150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe500cfe20 0x7efe500d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46368/0 (socket says 172.21.15.138:46368) 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe500ce7c0 0x7efe50160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42616/0 (socket says 172.21.15.138:42616) 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4ffff640 1 -- 172.21.15.138:0/3116934687 learned_addr learned my addr 172.21.15.138:0/3116934687 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4f7fe640 1 -- 172.21.15.138:0/3116934687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe500ce7c0 msgr2=0x7efe50160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4f7fe640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe500ce7c0 0x7efe50160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4f7fe640 1 -- 172.21.15.138:0/3116934687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe500cfe20 msgr2=0x7efe500d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4f7fe640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe500cfe20 0x7efe500d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4f7fe640 1 -- 172.21.15.138:0/3116934687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50150640 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe4f7fe640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5009c330 0x7efe50150100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe44031000 tx=0x7efe440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44041030 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4403d4e0 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe501605f0 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44037040 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe50160310 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe44044070 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.475+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7efe44046b20 con 0x7efe5009c330 2024-04-25T16:26:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.476+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5006a180 con 0x7efe5009c330 2024-04-25T16:26:32.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.479+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efe4404b070 con 0x7efe5009c330 2024-04-25T16:26:32.529 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:32.528+0000 7f61a583d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:26:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.551+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4b9e445a-59c0-4623-b982-7c8f29b19a53", "id": 3} v 0) -- 0x7efe500b0ab0 con 0x7efe5009c330 2024-04-25T16:26:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.553+0000 7efe37fff640 1 -- 172.21.15.138:0/3116934687 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4b9e445a-59c0-4623-b982-7c8f29b19a53", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7efe44046690 con 0x7efe5009c330 2024-04-25T16:26:32.554 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-04-25T16:26:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5009c330 msgr2=0x7efe50150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5009c330 0x7efe50150100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe44031000 tx=0x7efe440046c0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 shutdown_connections 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe500cfe20 0x7efe500d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe500ce7c0 0x7efe50160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 --2- 172.21.15.138:0/3116934687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5009c330 0x7efe50150100 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 >> 172.21.15.138:0/3116934687 conn(0x7efe500b7b60 msgr2=0x7efe500b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 shutdown_connections 2024-04-25T16:26:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.554+0000 7efe56070640 1 -- 172.21.15.138:0/3116934687 wait complete. 2024-04-25T16:26:32.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 2a101628-de9f-4b35-93a7-97d72c9bf949 4 2024-04-25T16:26:32.599 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:32.598+0000 7f0945796040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:26:32.616 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:32.615+0000 7f61a583d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:26:32.658 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:32.657+0000 7f61a583d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:26:32.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.673+0000 7fb0cd4a7640 1 Processor -- start 2024-04-25T16:26:32.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.673+0000 7fb0cd4a7640 1 -- start start 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.674+0000 7fb0cd4a7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b56a0 0x7fb0c80b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.674+0000 7fb0cd4a7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 0x7fb0c80d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.674+0000 7fb0cd4a7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c806c8c0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.674+0000 7fb0cd4a7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c806cba0 con 0x7fb0c80b9cc0 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.674+0000 7fb0cd4a7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c806ce80 con 0x7fb0c80b56a0 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c77fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 0x7fb0c80d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c77fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 0x7fb0c80d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46380/0 (socket says 172.21.15.138:46380) 2024-04-25T16:26:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c67fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b56a0 0x7fb0c80b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c77fe640 1 -- 172.21.15.138:0/2338463285 learned_addr learned my addr 172.21.15.138:0/2338463285 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c67fc640 1 -- 172.21.15.138:0/2338463285 >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 msgr2=0x7fb0c80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c67fc640 1 --2- 172.21.15.138:0/2338463285 >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 0x7fb0c80d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c67fc640 1 -- 172.21.15.138:0/2338463285 >> v1:172.21.15.94:6789/0 conn(0x7fb0c80d4aa0 legacy=0x7fb0c806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c67fc640 1 -- 172.21.15.138:0/2338463285 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c806d160 con 0x7fb0c80b56a0 2024-04-25T16:26:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.675+0000 7fb0c77fe640 1 --2- 172.21.15.138:0/2338463285 >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 0x7fb0c80d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.676+0000 7fb0c67fc640 1 --2- 172.21.15.138:0/2338463285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b56a0 0x7fb0c80b9780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80097a0 tx=0x7fb0b802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c943d144e1b42be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.676+0000 7fb0c5ffb640 1 -- 172.21.15.138:0/2338463285 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8037070 con 0x7fb0c80b56a0 2024-04-25T16:26:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.676+0000 7fb0c5ffb640 1 -- 172.21.15.138:0/2338463285 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b802adb0 con 0x7fb0c80b56a0 2024-04-25T16:26:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.676+0000 7fb0c5ffb640 1 -- 172.21.15.138:0/2338463285 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8032660 con 0x7fb0c80b56a0 2024-04-25T16:26:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/2338463285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b56a0 msgr2=0x7fb0c80b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/2338463285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b56a0 0x7fb0c80b9780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80097a0 tx=0x7fb0b802a060 comp rx=0 tx=0).stop 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/2338463285 shutdown_connections 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/2338463285 >> v2:172.21.15.138:3300/0 conn(0x7fb0c80b9cc0 0x7fb0c80d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/2338463285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b56a0 0x7fb0c80b9780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/2338463285 >> 172.21.15.138:0/2338463285 conn(0x7fb0c80b7e60 msgr2=0x7fb0c80b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/2338463285 shutdown_connections 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.677+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/2338463285 wait complete. 2024-04-25T16:26:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.679+0000 7fb0cd4a7640 1 Processor -- start 2024-04-25T16:26:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.679+0000 7fb0cd4a7640 1 -- start start 2024-04-25T16:26:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0cd4a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c80b56a0 0x7fb0c80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0cd4a7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 0x7fb0c8159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0cd4a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c80d4aa0 0x7fb0c815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0cd4a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c80ba0d0 con 0x7fb0c80b56a0 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0cd4a7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c806c570 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0cd4a7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c80ba250 con 0x7fb0c80b9cc0 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c67fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 0x7fb0c8159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c80d4aa0 0x7fb0c815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c67fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 0x7fb0c8159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42640/0 (socket says 172.21.15.138:42640) 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c67fc640 1 -- 172.21.15.138:0/3309131484 learned_addr learned my addr 172.21.15.138:0/3309131484 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c6ffd640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c80b56a0 0x7fb0c80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c77fe640 1 -- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 msgr2=0x7fb0c8159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c77fe640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 0x7fb0c8159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c77fe640 1 -- 172.21.15.138:0/3309131484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c80b56a0 msgr2=0x7fb0c80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c77fe640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c80b56a0 0x7fb0c80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c77fe640 1 -- 172.21.15.138:0/3309131484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c80c9f10 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.680+0000 7fb0c67fc640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 0x7fb0c8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0c77fe640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c80d4aa0 0x7fb0c815f270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009db0 tx=0x7fb0c000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000cb20 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c000cce0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c80cb8d0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003bd0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c80cbe90 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0c001b030 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.681+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fb0c001dba0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.682+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c80b87a0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.684+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb0c80b87a0 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.693 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:32.692+0000 7f0945796040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:26:32.737 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:32.737+0000 7f0945796040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:26:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.756+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a101628-de9f-4b35-93a7-97d72c9bf949", "id": 4} v 0) -- 0x7fb0c806a180 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.760+0000 7fb0af7fe640 1 -- 172.21.15.138:0/3309131484 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a101628-de9f-4b35-93a7-97d72c9bf949", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fb0c0022020 con 0x7fb0c80d4aa0 2024-04-25T16:26:32.761 INFO:teuthology.orchestra.run.smithi138.stdout:4 2024-04-25T16:26:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c80d4aa0 msgr2=0x7fb0c815f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c80d4aa0 0x7fb0c815f270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009db0 tx=0x7fb0c000c490 comp rx=0 tx=0).stop 2024-04-25T16:26:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 shutdown_connections 2024-04-25T16:26:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c80d4aa0 0x7fb0c815f270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c80b9cc0 0x7fb0c8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 --2- 172.21.15.138:0/3309131484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c80b56a0 0x7fb0c80cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 >> 172.21.15.138:0/3309131484 conn(0x7fb0c80b7e60 msgr2=0x7fb0c80ba940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 shutdown_connections 2024-04-25T16:26:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.761+0000 7fb0cd4a7640 1 -- 172.21.15.138:0/3309131484 wait complete. 2024-04-25T16:26:32.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 9e2df0c1-56a4-4213-b625-c803b52fd641 5 2024-04-25T16:26:32.822 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:32.822+0000 7f61a583d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:26:32.869 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:32.869+0000 7f61a583d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:26:32.919 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:32.918+0000 7f0945796040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:26:32.922 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:32.921+0000 7f61a583d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:26:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.924+0000 7fe6f5030640 1 Processor -- start 2024-04-25T16:26:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.924+0000 7fe6f5030640 1 -- start start 2024-04-25T16:26:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6f5030640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe6f00b9510 0x7fe6f00b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6f5030640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6f00cf690 0x7fe6f00d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6f5030640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f00d1220 con 0x7fe6f00d0e10 2024-04-25T16:26:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6f5030640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f00d01c0 con 0x7fe6f00b9510 2024-04-25T16:26:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6f5030640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f00cfee0 con 0x7fe6f00cf690 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6e7fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6f00cf690 0x7fe6f00d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6e7fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6f00cf690 0x7fe6f00d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46406/0 (socket says 172.21.15.138:46406) 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2557181171 learned_addr learned my addr 172.21.15.138:0/2557181171 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6eed9d640 1 --2- 172.21.15.138:0/2557181171 >> v2:172.21.15.94:3300/0 conn(0x7fe6f00b9510 0x7fe6f00b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2557181171 >> v2:172.21.15.94:3300/0 conn(0x7fe6f00b9510 msgr2=0x7fe6f00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.925+0000 7fe6e7fff640 1 --2- 172.21.15.138:0/2557181171 >> v2:172.21.15.94:3300/0 conn(0x7fe6f00b9510 0x7fe6f00b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.926+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2557181171 >> v1:172.21.15.94:6789/0 conn(0x7fe6f00d0e10 legacy=0x7fe6f00d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.926+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2557181171 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f0071770 con 0x7fe6f00cf690 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.926+0000 7fe6e7fff640 1 --2- 172.21.15.138:0/2557181171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f00d08d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00cead0 tx=0x7fe6d802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd3aca8848cd357 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.926+0000 7fe6ee59c640 1 -- 172.21.15.138:0/2557181171 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8037070 con 0x7fe6f00cf690 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.926+0000 7fe6ee59c640 1 -- 172.21.15.138:0/2557181171 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8032440 con 0x7fe6f00cf690 2024-04-25T16:26:32.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.926+0000 7fe6ee59c640 1 -- 172.21.15.138:0/2557181171 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8032810 con 0x7fe6f00cf690 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 -- 172.21.15.138:0/2557181171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 msgr2=0x7fe6f00d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2557181171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f00d08d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00cead0 tx=0x7fe6d802a270 comp rx=0 tx=0).stop 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 -- 172.21.15.138:0/2557181171 shutdown_connections 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2557181171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f00d08d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2557181171 >> v2:172.21.15.94:3300/0 conn(0x7fe6f00b9510 0x7fe6f00b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 -- 172.21.15.138:0/2557181171 >> 172.21.15.138:0/2557181171 conn(0x7fe6f00d3f30 msgr2=0x7fe6f00d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 -- 172.21.15.138:0/2557181171 shutdown_connections 2024-04-25T16:26:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.927+0000 7fe6f5030640 1 -- 172.21.15.138:0/2557181171 wait complete. 2024-04-25T16:26:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.929+0000 7fe6f5030640 1 Processor -- start 2024-04-25T16:26:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.929+0000 7fe6f5030640 1 -- start start 2024-04-25T16:26:32.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.930+0000 7fe6f5030640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f00b9510 0x7fe6f015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.930+0000 7fe6f5030640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f0154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.930+0000 7fe6f5030640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6f00d0e10 0x7fe6f00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.930+0000 7fe6f5030640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f00cfaa0 con 0x7fe6f00b9510 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.930+0000 7fe6f5030640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f0071d30 con 0x7fe6f00cf690 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.930+0000 7fe6f5030640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f00cfc20 con 0x7fe6f00d0e10 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f0154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f0154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46418/0 (socket says 172.21.15.138:46418) 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2867513671 learned_addr learned my addr 172.21.15.138:0/2867513671 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6ef59e640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6f00d0e10 0x7fe6f00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6eed9d640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f00b9510 0x7fe6f015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6f00d0e10 msgr2=0x7fe6f00b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6f00d0e10 0x7fe6f00b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2867513671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f00b9510 msgr2=0x7fe6f015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f00b9510 0x7fe6f015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 -- 172.21.15.138:0/2867513671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f006e5d0 con 0x7fe6f00cf690 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6ef59e640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6f00d0e10 0x7fe6f00b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e7fff640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f0154620 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8037040 con 0x7fe6f00cf690 2024-04-25T16:26:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8042070 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f006e8b0 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8030880 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f015ce70 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.931+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe6d8030a20 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.932+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe6d8047b90 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.932+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f006a180 con 0x7fe6f00cf690 2024-04-25T16:26:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:32.935+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe6d8047d90 con 0x7fe6f00cf690 2024-04-25T16:26:32.973 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:32.972+0000 7f0945796040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:26:33.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.007+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9e2df0c1-56a4-4213-b625-c803b52fd641", "id": 5} v 0) -- 0x7fe6f00b6f90 con 0x7fe6f00cf690 2024-04-25T16:26:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.013+0000 7fe6e77fe640 1 -- 172.21.15.138:0/2867513671 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9e2df0c1-56a4-4213-b625-c803b52fd641", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe6d8030e00 con 0x7fe6f00cf690 2024-04-25T16:26:33.014 INFO:teuthology.orchestra.run.smithi138.stdout:5 2024-04-25T16:26:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 msgr2=0x7fe6f0154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f0154620 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d80046c0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 shutdown_connections 2024-04-25T16:26:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6f00d0e10 0x7fe6f00b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f00cf690 0x7fe6f0154620 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 --2- 172.21.15.138:0/2867513671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f00b9510 0x7fe6f015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 >> 172.21.15.138:0/2867513671 conn(0x7fe6f00d3f30 msgr2=0x7fe6f00b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 shutdown_connections 2024-04-25T16:26:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.014+0000 7fe6f5030640 1 -- 172.21.15.138:0/2867513671 wait complete. 2024-04-25T16:26:33.025 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 5fcc96fa-a293-459a-bc53-feeac42e222c 6 2024-04-25T16:26:33.029 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:33.028+0000 7f0945796040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:26:33.034 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:33.033+0000 7f61a583d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:26:33.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.132+0000 7f587ec27640 1 Processor -- start 2024-04-25T16:26:33.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.132+0000 7f587ec27640 1 -- start start 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587ec27640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f58780b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587ec27640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f58780cf140 0x7f58780d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587ec27640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587806f840 con 0x7f58780d3890 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587ec27640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f58780d0830 con 0x7f58780cf140 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587ec27640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f58780b2810 con 0x7f58780cfcf0 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f58780b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f58780b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42672/0 (socket says 172.21.15.138:42672) 2024-04-25T16:26:33.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 -- 172.21.15.138:0/1755260609 learned_addr learned my addr 172.21.15.138:0/1755260609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587d1c4640 1 --2- 172.21.15.138:0/1755260609 >> v2:172.21.15.138:3300/0 conn(0x7f58780cf140 0x7f58780d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 -- 172.21.15.138:0/1755260609 >> v2:172.21.15.138:3300/0 conn(0x7f58780cf140 msgr2=0x7f58780d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 --2- 172.21.15.138:0/1755260609 >> v2:172.21.15.138:3300/0 conn(0x7f58780cf140 0x7f58780d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 -- 172.21.15.138:0/1755260609 >> v1:172.21.15.94:6789/0 conn(0x7f58780d3890 legacy=0x7f58780cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f5877fff640 1 -- 172.21.15.138:0/1755260609 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58780b2af0 con 0x7f58780cfcf0 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.134+0000 7f587d1c4640 1 --2- 172.21.15.138:0/1755260609 >> v2:172.21.15.138:3300/0 conn(0x7f58780cf140 0x7f58780d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.135+0000 7f5877fff640 1 --2- 172.21.15.138:0/1755260609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f58780b38b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5864009870 tx=0x7f586402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8f9d005cdc6ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.135+0000 7f58777fe640 1 -- 172.21.15.138:0/1755260609 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864037070 con 0x7f58780cfcf0 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.135+0000 7f58777fe640 1 -- 172.21.15.138:0/1755260609 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5864032440 con 0x7f58780cfcf0 2024-04-25T16:26:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.135+0000 7f58777fe640 1 -- 172.21.15.138:0/1755260609 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864032810 con 0x7f58780cfcf0 2024-04-25T16:26:33.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 -- 172.21.15.138:0/1755260609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 msgr2=0x7f58780b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 --2- 172.21.15.138:0/1755260609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f58780b38b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5864009870 tx=0x7f586402a270 comp rx=0 tx=0).stop 2024-04-25T16:26:33.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 -- 172.21.15.138:0/1755260609 shutdown_connections 2024-04-25T16:26:33.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 --2- 172.21.15.138:0/1755260609 >> v2:172.21.15.138:3300/0 conn(0x7f58780cf140 0x7f58780d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 --2- 172.21.15.138:0/1755260609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f58780b38b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 -- 172.21.15.138:0/1755260609 >> 172.21.15.138:0/1755260609 conn(0x7f58780b7b20 msgr2=0x7f58780b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 -- 172.21.15.138:0/1755260609 shutdown_connections 2024-04-25T16:26:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.136+0000 7f587ec27640 1 -- 172.21.15.138:0/1755260609 wait complete. 2024-04-25T16:26:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.137+0000 7f587ec27640 1 Processor -- start 2024-04-25T16:26:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.138+0000 7f587ec27640 1 -- start start 2024-04-25T16:26:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587ec27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58780cf140 0x7f58780cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587ec27640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f587815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587ec27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58780d3890 0x7f5878161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587ec27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58780b3390 con 0x7f58780cf140 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587ec27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58780b30b0 con 0x7f58780d3890 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587ec27640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f58780b3f80 con 0x7f58780cfcf0 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f587815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587d1c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58780d3890 0x7f5878161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f587815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42674/0 (socket says 172.21.15.138:42674) 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 -- 172.21.15.138:0/3100197630 learned_addr learned my addr 172.21.15.138:0/3100197630 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587c9c3640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58780cf140 0x7f58780cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 -- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58780d3890 msgr2=0x7f5878161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58780d3890 0x7f5878161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 -- 172.21.15.138:0/3100197630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58780cf140 msgr2=0x7f58780cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58780cf140 0x7f58780cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f5877fff640 1 -- 172.21.15.138:0/3100197630 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587815fb20 con 0x7f58780cfcf0 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.139+0000 7f587d1c4640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58780d3890 0x7f5878161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f5877fff640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f587815f5e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5864000c00 tx=0x7f58640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864037040 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5864042070 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864030880 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58780c8590 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58780c8b50 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5864030a20 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.140+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f5864047b40 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.141+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587806a180 con 0x7f58780cfcf0 2024-04-25T16:26:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.143+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5864047e20 con 0x7f58780cfcf0 2024-04-25T16:26:33.150 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:33.150+0000 7f0945796040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:26:33.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.216+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5fcc96fa-a293-459a-bc53-feeac42e222c", "id": 6} v 0) -- 0x7f58780cfad0 con 0x7f58780cfcf0 2024-04-25T16:26:33.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.223+0000 7f58757fa640 1 -- 172.21.15.138:0/3100197630 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5fcc96fa-a293-459a-bc53-feeac42e222c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f5864047460 con 0x7f58780cfcf0 2024-04-25T16:26:33.224 INFO:teuthology.orchestra.run.smithi138.stdout:6 2024-04-25T16:26:33.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 msgr2=0x7f587815f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f587815f5e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5864000c00 tx=0x7f58640046c0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 shutdown_connections 2024-04-25T16:26:33.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58780d3890 0x7f5878161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58780cfcf0 0x7f587815f5e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 --2- 172.21.15.138:0/3100197630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58780cf140 0x7f58780cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 >> 172.21.15.138:0/3100197630 conn(0x7f58780b7b20 msgr2=0x7f587806ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 shutdown_connections 2024-04-25T16:26:33.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.224+0000 7f587ec27640 1 -- 172.21.15.138:0/3100197630 wait complete. 2024-04-25T16:26:33.235 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 6377ea75-413c-4635-baf1-69b26cc937b9 7 2024-04-25T16:26:33.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.383+0000 7ff91baaf640 1 Processor -- start 2024-04-25T16:26:33.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.383+0000 7ff91baaf640 1 -- start start 2024-04-25T16:26:33.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91baaf640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9140705d0 0x7ff9140b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91baaf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff9140d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91baaf640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff914160d00 con 0x7ff91409bc20 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91baaf640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff9140d1ef0 con 0x7ff9140705d0 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91baaf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff914070ab0 con 0x7ff9140d2360 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91984b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9140705d0 0x7ff9140b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91a04c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff9140d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91984b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9140705d0 0x7ff9140b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46438/0 (socket says 172.21.15.138:46438) 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91a04c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff9140d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42684/0 (socket says 172.21.15.138:42684) 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.384+0000 7ff91984b640 1 -- 172.21.15.138:0/1830418308 learned_addr learned my addr 172.21.15.138:0/1830418308 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.385+0000 7ff91984b640 1 -- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 msgr2=0x7ff9140d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.385+0000 7ff91984b640 1 --2- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff9140d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.385+0000 7ff91984b640 1 -- 172.21.15.138:0/1830418308 >> v1:172.21.15.138:6789/0 conn(0x7ff91409bc20 legacy=0x7ff91409c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.385+0000 7ff91984b640 1 -- 172.21.15.138:0/1830418308 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff914070d90 con 0x7ff9140705d0 2024-04-25T16:26:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.385+0000 7ff91a04c640 1 --2- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff9140d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:33.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.385+0000 7ff91984b640 1 --2- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b1fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff910009870 tx=0x7ff91002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5eb2bd46d6b30b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:33.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff918849640 1 -- 172.21.15.138:0/1830418308 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910037070 con 0x7ff9140705d0 2024-04-25T16:26:33.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff918849640 1 -- 172.21.15.138:0/1830418308 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff910032440 con 0x7ff9140705d0 2024-04-25T16:26:33.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff918849640 1 -- 172.21.15.138:0/1830418308 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910032810 con 0x7ff9140705d0 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff91baaf640 1 -- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 msgr2=0x7ff9140b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff91baaf640 1 --2- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b1fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff910009870 tx=0x7ff91002a270 comp rx=0 tx=0).stop 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff91baaf640 1 -- 172.21.15.138:0/1830418308 shutdown_connections 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff91baaf640 1 --2- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff9140d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff91baaf640 1 --2- 172.21.15.138:0/1830418308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b1fe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.386+0000 7ff91baaf640 1 -- 172.21.15.138:0/1830418308 >> 172.21.15.138:0/1830418308 conn(0x7ff9140b7b60 msgr2=0x7ff9140b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.387+0000 7ff91baaf640 1 -- 172.21.15.138:0/1830418308 shutdown_connections 2024-04-25T16:26:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.387+0000 7ff91baaf640 1 -- 172.21.15.138:0/1830418308 wait complete. 2024-04-25T16:26:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.388+0000 7ff91baaf640 1 Processor -- start 2024-04-25T16:26:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.388+0000 7ff91baaf640 1 -- start start 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.389+0000 7ff91baaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91409bc20 0x7ff91414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff91415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9140cf040 con 0x7ff91409bc20 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff914071350 con 0x7ff9140705d0 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff9140b2520 con 0x7ff9140d2360 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91a04c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff91415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91a04c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff91415ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:42686/0 (socket says 172.21.15.138:42686) 2024-04-25T16:26:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91a04c640 1 -- 172.21.15.138:0/2287528919 learned_addr learned my addr 172.21.15.138:0/2287528919 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91904a640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91409bc20 0x7ff91414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 -- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 msgr2=0x7ff91415ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff91415ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 -- 172.21.15.138:0/2287528919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91409bc20 msgr2=0x7ff91414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91409bc20 0x7ff91414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 -- 172.21.15.138:0/2287528919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9140b0b00 con 0x7ff9140705d0 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91984b640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b0040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff910031000 tx=0x7ff9100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910041020 con 0x7ff9140705d0 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff910030660 con 0x7ff9140705d0 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9140b0de0 con 0x7ff9140705d0 2024-04-25T16:26:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910037040 con 0x7ff9140705d0 2024-04-25T16:26:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.390+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9140b13a0 con 0x7ff9140705d0 2024-04-25T16:26:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.391+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff910044020 con 0x7ff9140705d0 2024-04-25T16:26:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.391+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff91003dba0 con 0x7ff9140705d0 2024-04-25T16:26:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.392+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91406a180 con 0x7ff9140705d0 2024-04-25T16:26:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.394+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff91004b030 con 0x7ff9140705d0 2024-04-25T16:26:33.413 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:33.412+0000 7f61a583d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:26:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.466+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6377ea75-413c-4635-baf1-69b26cc937b9", "id": 7} v 0) -- 0x7ff9140d3ac0 con 0x7ff9140705d0 2024-04-25T16:26:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.472+0000 7ff90a7fc640 1 -- 172.21.15.138:0/2287528919 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6377ea75-413c-4635-baf1-69b26cc937b9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff91003d420 con 0x7ff9140705d0 2024-04-25T16:26:33.473 INFO:teuthology.orchestra.run.smithi138.stdout:7 2024-04-25T16:26:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 msgr2=0x7ff9140b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b0040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff910031000 tx=0x7ff9100046c0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 shutdown_connections 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9140d2360 0x7ff91415ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91409bc20 0x7ff91414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 --2- 172.21.15.138:0/2287528919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9140705d0 0x7ff9140b0040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 >> 172.21.15.138:0/2287528919 conn(0x7ff9140b7b60 msgr2=0x7ff9140cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 shutdown_connections 2024-04-25T16:26:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:26:33.473+0000 7ff91baaf640 1 -- 172.21.15.138:0/2287528919 wait complete. 2024-04-25T16:26:33.485 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T16:26:33.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T16:26:33.487 INFO:tasks.ceph.osd.0:Started 2024-04-25T16:26:33.487 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T16:26:33.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T16:26:33.489 INFO:tasks.ceph.osd.1:Started 2024-04-25T16:26:33.490 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T16:26:33.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T16:26:33.492 INFO:tasks.ceph.osd.2:Started 2024-04-25T16:26:33.492 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T16:26:33.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T16:26:33.494 INFO:tasks.ceph.osd.3:Started 2024-04-25T16:26:33.494 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T16:26:33.494 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T16:26:33.496 INFO:tasks.ceph.osd.4:Started 2024-04-25T16:26:33.496 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T16:26:33.497 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T16:26:33.498 INFO:tasks.ceph.osd.5:Started 2024-04-25T16:26:33.499 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T16:26:33.499 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T16:26:33.500 INFO:tasks.ceph.osd.6:Started 2024-04-25T16:26:33.501 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T16:26:33.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T16:26:33.505 INFO:tasks.ceph.osd.7:Started 2024-04-25T16:26:33.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:26:33.559 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:33.558+0000 7f0945796040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:26:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.640+0000 7f8d58ce0640 1 Processor -- start 2024-04-25T16:26:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.640+0000 7f8d58ce0640 1 -- start start 2024-04-25T16:26:33.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.641+0000 7f8d58ce0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d5406ce00 0x7f8d54145950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.641+0000 7f8d58ce0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5406d500 con 0x7f8d540d3480 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.641+0000 7f8d58ce0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d54162190 con 0x7f8d5406f4c0 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.641+0000 7f8d58ce0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5406f8d0 con 0x7f8d5406ce00 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.642+0000 7f8d52d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d5406ce00 0x7f8d54145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.642+0000 7f8d4bfff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8d540d3480 0x7f8d54145240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48704/0 (socket says 172.21.15.94:48704) 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.642+0000 7f8d4bfff640 1 -- 172.21.15.94:0/1467388870 learned_addr learned my addr 172.21.15.94:0/1467388870 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.642+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1467388870 >> v1:172.21.15.138:6789/0 conn(0x7f8d5406f4c0 legacy=0x7f8d540d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.642+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1467388870 >> v1:172.21.15.94:6789/0 conn(0x7f8d540d3480 legacy=0x7f8d54145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.642+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1467388870 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d54162470 con 0x7f8d5406ce00 2024-04-25T16:26:33.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.643+0000 7f8d52d9d640 1 --2- 172.21.15.94:0/1467388870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406ce00 0x7f8d54145950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4729b8c2ed8026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:33.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.643+0000 7f8d51d9b640 1 -- 172.21.15.94:0/1467388870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c007740 con 0x7f8d5406ce00 2024-04-25T16:26:33.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.643+0000 7f8d51d9b640 1 -- 172.21.15.94:0/1467388870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0078e0 con 0x7f8d5406ce00 2024-04-25T16:26:33.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.643+0000 7f8d51d9b640 1 -- 172.21.15.94:0/1467388870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c007cb0 con 0x7f8d5406ce00 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1467388870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406ce00 msgr2=0x7f8d54145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 --2- 172.21.15.94:0/1467388870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406ce00 0x7f8d54145950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c02a610 comp rx=0 tx=0).stop 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1467388870 shutdown_connections 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 --2- 172.21.15.94:0/1467388870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406ce00 0x7f8d54145950 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1467388870 >> 172.21.15.94:0/1467388870 conn(0x7f8d540b7cc0 msgr2=0x7f8d540b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1467388870 shutdown_connections 2024-04-25T16:26:33.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.644+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1467388870 wait complete. 2024-04-25T16:26:33.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.647+0000 7f8d58ce0640 1 Processor -- start 2024-04-25T16:26:33.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.647+0000 7f8d58ce0640 1 -- start start 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d58ce0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d5406ce00 0x7f8d541375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d58ce0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d5406f4c0 0x7f8d54136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d58ce0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 0x7f8d54136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d58ce0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5406d210 con 0x7f8d54137b30 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d58ce0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54162ac0 con 0x7f8d5406f4c0 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d58ce0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d54161c70 con 0x7f8d5406ce00 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d52d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 0x7f8d54136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d52d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 0x7f8d54136b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38064/0 (socket says 172.21.15.94:38064) 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1219074706 learned_addr learned my addr 172.21.15.94:0/1219074706 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d4bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d5406f4c0 0x7f8d54136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d5259c640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d5406ce00 0x7f8d541375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d5406ce00 msgr2=0x7f8d541375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.649+0000 7f8d52d9d640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d5406ce00 0x7f8d541375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d5406f4c0 msgr2=0x7f8d54136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d52d9d640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d5406f4c0 0x7f8d54136650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d52d9d640 1 -- 172.21.15.94:0/1219074706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d54139690 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d5259c640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d5406ce00 0x7f8d541375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d52d9d640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 0x7f8d54136b90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009eb0 tx=0x7f8d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c03f070 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4c031c40 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d54135b70 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c003990 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5415d060 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d4c004030 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.650+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8d4c03bbb0 con 0x7f8d54137b30 2024-04-25T16:26:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.651+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5406a180 con 0x7f8d54137b30 2024-04-25T16:26:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.653+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8d4c004500 con 0x7f8d54137b30 2024-04-25T16:26:33.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.713+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d5409ca10 con 0x7f8d54137b30 2024-04-25T16:26:33.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.713+0000 7f8d4affd640 1 -- 172.21.15.94:0/1219074706 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f8d4c0046e0 con 0x7f8d54137b30 2024-04-25T16:26:33.714 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:26:33.715 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:26:33.468939+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:26:33.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 msgr2=0x7f8d54136b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 0x7f8d54136b90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009eb0 tx=0x7f8d4c004a40 comp rx=0 tx=0).stop 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 shutdown_connections 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54137b30 0x7f8d54136b90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d5406f4c0 0x7f8d54136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 --2- 172.21.15.94:0/1219074706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d5406ce00 0x7f8d541375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.714+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 >> 172.21.15.94:0/1219074706 conn(0x7f8d540b7cc0 msgr2=0x7f8d5413d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.715+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 shutdown_connections 2024-04-25T16:26:33.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:33.715+0000 7f8d58ce0640 1 -- 172.21.15.94:0/1219074706 wait complete. 2024-04-25T16:26:33.722 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:33.722+0000 7f61a583d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:26:33.726 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T16:26:33.726 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T16:26:33.777 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:33.776+0000 7f61a583d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:26:33.845 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:33.844+0000 7f0945796040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:26:33.894 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:33.893+0000 7f61a583d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:26:33.898 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:33.897+0000 7f0945796040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:26:34.015 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.014+0000 7f0945796040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:26:34.108 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:26:34.107+0000 7fb81d210640 -1 Falling back to public interface 2024-04-25T16:26:34.109 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:26:34.108+0000 7f61bd3d6640 -1 Falling back to public interface 2024-04-25T16:26:34.122 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:26:34.121+0000 7fde22e5d640 -1 Falling back to public interface 2024-04-25T16:26:34.122 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:26:34.121+0000 7f2db462d640 -1 Falling back to public interface 2024-04-25T16:26:34.127 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:26:34.126+0000 7efdadfbc640 -1 Falling back to public interface 2024-04-25T16:26:34.134 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:26:34.134+0000 7f6cd5553640 -1 Falling back to public interface 2024-04-25T16:26:34.139 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:26:34.138+0000 7f9e6d419640 -1 Falling back to public interface 2024-04-25T16:26:34.151 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:26:34.150+0000 7f21a3c43640 -1 Falling back to public interface 2024-04-25T16:26:34.177 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.177+0000 7f61a583d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:26:34.226 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.225+0000 7f61a583d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:26:34.298 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.297+0000 7f0945796040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:26:34.346 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.345+0000 7f0945796040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:26:34.380 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.379+0000 7f61a583d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:26:34.505 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.504+0000 7f0945796040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:26:34.516 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.515+0000 7f61a583d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:26:34.532 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T16:26:34.567 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.566+0000 7f61a583d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:26:34.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.632+0000 7f39473c5640 1 Processor -- start 2024-04-25T16:26:34.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.632+0000 7f39473c5640 1 -- start start 2024-04-25T16:26:34.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.633+0000 7f39473c5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f39400d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f39473c5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39400d2e90 con 0x7f394006d1f0 2024-04-25T16:26:34.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f39473c5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f394006c710 con 0x7f39400d1820 2024-04-25T16:26:34.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f39473c5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39400d3150 con 0x7f39400b7c30 2024-04-25T16:26:34.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3945962640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f394006d1f0 0x7f394006d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48714/0 (socket says 172.21.15.94:48714) 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3945962640 1 -- 172.21.15.94:0/2404986899 learned_addr learned my addr 172.21.15.94:0/2404986899 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3945161640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f39400d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692564246 0 0) 0x7f39400d2e90 con 0x7f394006d1f0 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f392c003520 con 0x7f394006d1f0 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558500871 0 0) 0x7f394006c710 con 0x7f39400d1820 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39400d2e90 con 0x7f39400d1820 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1481687860 0 0) 0x7f392c003520 con 0x7f394006d1f0 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 msgr2=0x7f39400d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f39400d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 >> v1:172.21.15.138:6789/0 conn(0x7f39400d1820 legacy=0x7f39400d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394006c9f0 con 0x7f394006d1f0 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f39380019f0 con 0x7f394006d1f0 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3945161640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f39400d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:34.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 >> v1:172.21.15.94:6789/0 conn(0x7f394006d1f0 legacy=0x7f394006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c003b70 0x7f392c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c004800 0x7f392c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f392c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39400d3150 con 0x7f392c003b70 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f392c00bad0 con 0x7f392c004800 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f392c00bcd0 con 0x7f39400b7c30 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.634+0000 7f3945161640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c003b70 0x7f392c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3944960640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c004800 0x7f392c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945962640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f392c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945161640 1 -- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 msgr2=0x7f392c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945161640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400b7c30 0x7f392c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945161640 1 -- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c004800 msgr2=0x7f392c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945161640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c004800 0x7f392c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945161640 1 -- 172.21.15.94:0/2404986899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c003520 con 0x7f392c003b70 2024-04-25T16:26:34.639 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.635+0000 7f3945161640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c003b70 0x7f392c003fc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3930009c90 tx=0x7f392c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b791d834b1ce67cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930036070 con 0x7f392c003b70 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3930031400 con 0x7f392c003b70 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f3937fff640 1 -- 172.21.15.94:0/2404986899 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930009860 con 0x7f392c003b70 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f39473c5640 1 -- 172.21.15.94:0/2404986899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c003b70 msgr2=0x7f392c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f39473c5640 1 --2- 172.21.15.94:0/2404986899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c003b70 0x7f392c003fc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3930009c90 tx=0x7f392c005830 comp rx=0 tx=0).stop 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f3945962640 1 -- 172.21.15.94:0/2404986899 reap_dead start 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f39473c5640 1 -- 172.21.15.94:0/2404986899 shutdown_connections 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f39473c5640 1 -- 172.21.15.94:0/2404986899 >> 172.21.15.94:0/2404986899 conn(0x7f394009b9b0 msgr2=0x7f39400b7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f39473c5640 1 -- 172.21.15.94:0/2404986899 shutdown_connections 2024-04-25T16:26:34.640 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.636+0000 7f39473c5640 1 -- 172.21.15.94:0/2404986899 wait complete. 2024-04-25T16:26:34.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.638+0000 7f39473c5640 1 Processor -- start 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.638+0000 7f39473c5640 1 -- start start 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.639+0000 7f39473c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c003b70 0x7f394014a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.639+0000 7f39473c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 0x7f39400ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.639+0000 7f39473c5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400cf090 0x7f39401e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.639+0000 7f39473c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39400d2560 con 0x7f3940161630 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.639+0000 7f39473c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3940160b90 con 0x7f392c003b70 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.639+0000 7f39473c5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39400d26e0 con 0x7f39400cf090 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 0x7f39400ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 0x7f39400ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38088/0 (socket says 172.21.15.94:38088) 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 -- 172.21.15.94:0/772340363 learned_addr learned my addr 172.21.15.94:0/772340363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3945161640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c003b70 0x7f394014a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3945962640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400cf090 0x7f39401e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 -- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400cf090 msgr2=0x7f39401e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:34.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400cf090 0x7f39401e8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 -- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c003b70 msgr2=0x7f394014a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c003b70 0x7f394014a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 -- 172.21.15.94:0/772340363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39401f6e40 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3944960640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 0x7f39400ceb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f393c00f7f0 tx=0x7f393c00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c01a070 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393c005320 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394014b000 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c00fea0 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.640+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394014b5c0 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.641+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f393c01d070 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.641+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f393c006e00 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.641+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394009c130 con 0x7f3940161630 2024-04-25T16:26:34.644 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.642+0000 7f61a583d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:26:34.645 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.640+0000 7f0945796040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:26:34.646 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.645+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f394009c130 con 0x7f3940161630 2024-04-25T16:26:34.691 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:34.691+0000 7f61a583d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:26:34.692 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.691+0000 7f0945796040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:26:34.707 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.706+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39400708f0 con 0x7f3940161630 2024-04-25T16:26:34.708 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.707+0000 7f3935ffb640 1 -- 172.21.15.94:0/772340363 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f393c006830 con 0x7f3940161630 2024-04-25T16:26:34.708 INFO:teuthology.misc.health.smithi094.stdout: 2024-04-25T16:26:34.708 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":10,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:26:33.468939+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:26:34.709 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 msgr2=0x7f39400ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:34.709 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 0x7f39400ceb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f393c00f7f0 tx=0x7f393c00fcb0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.709 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 shutdown_connections 2024-04-25T16:26:34.709 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39400cf090 0x7f39401e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.710 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3940161630 0x7f39400ceb50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.710 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 --2- 172.21.15.94:0/772340363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f392c003b70 0x7f394014a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:34.710 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 >> 172.21.15.94:0/772340363 conn(0x7f394009b9b0 msgr2=0x7f394006fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:34.710 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 shutdown_connections 2024-04-25T16:26:34.710 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:34.708+0000 7f39473c5640 1 -- 172.21.15.94:0/772340363 wait complete. 2024-04-25T16:26:34.719 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T16:26:34.763 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.762+0000 7f0945796040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:26:34.808 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:34.807+0000 7f0945796040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:26:35.167 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:35.166+0000 7f61a583d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:26:35.285 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:35.284+0000 7f61a583d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:26:35.288 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:35.287+0000 7f0945796040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:26:35.411 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:26:35.410+0000 7f0945796040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:26:36.300 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:36.300+0000 7f61451ee640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T16:26:36.301 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:26:36.300+0000 7f61451ee640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T16:26:37.552 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:26:37.552+0000 7f61bd3d6640 -1 osd.1 0 log_to_monitors true 2024-04-25T16:26:37.591 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:26:37.590+0000 7fb81d210640 -1 osd.6 0 log_to_monitors true 2024-04-25T16:26:37.611 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:26:37.610+0000 7efdadfbc640 -1 osd.7 0 log_to_monitors true 2024-04-25T16:26:37.611 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:26:37.611+0000 7f2db462d640 -1 osd.0 0 log_to_monitors true 2024-04-25T16:26:37.634 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:26:37.634+0000 7fde22e5d640 -1 osd.3 0 log_to_monitors true 2024-04-25T16:26:37.643 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:26:37.642+0000 7f9e6d419640 -1 osd.4 0 log_to_monitors true 2024-04-25T16:26:37.654 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:26:37.653+0000 7f21a3c43640 -1 osd.5 0 log_to_monitors true 2024-04-25T16:26:37.655 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:26:37.656+0000 7f6cd5553640 -1 osd.2 0 log_to_monitors true 2024-04-25T16:26:40.729 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:26:40.728+0000 7f2dafb78640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T16:26:40.735 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:26:40.734+0000 7f61b936c640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T16:26:40.741 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:26:40.741+0000 7f2da7973640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:40.748 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:26:40.748+0000 7f61b1167640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:40.838 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:26:40.837+0000 7f6cd14e9640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T16:26:40.850 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:26:40.850+0000 7fde1edf3640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T16:26:40.852 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:26:40.852+0000 7f6cc92e4640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:40.862 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:26:40.862+0000 7fde16bee640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:41.525 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T16:26:41.556 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:26:41.555+0000 7f9e69520640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T16:26:41.584 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:26:41.583+0000 7f9e60b07640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:41.606 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:26:41.606+0000 7f219fbd9640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T16:26:41.622 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:26:41.621+0000 7f21979d4640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:41.631 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.630+0000 7f8ca8b9d640 1 Processor -- start 2024-04-25T16:26:41.631 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.630+0000 7f8ca8b9d640 1 -- start start 2024-04-25T16:26:41.631 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:26:41.630+0000 7fb818b04640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T16:26:41.632 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca8b9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8ca40d0120 0x7f8ca4164420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca8b9d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca4164d70 con 0x7f8ca40b2eb0 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca8b9d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca40b4c80 con 0x7f8ca4164960 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca8b9d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca40b2310 con 0x7f8ca40d0120 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca37fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8ca40d0120 0x7f8ca4164420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca3fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8ca40b2eb0 0x7f8ca40b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46422/0 (socket says 172.21.15.94:46422) 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.631+0000 7f8ca3fff640 1 -- 172.21.15.94:0/3361829144 learned_addr learned my addr 172.21.15.94:0/3361829144 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.632+0000 7f8ca37fe640 1 -- 172.21.15.94:0/3361829144 >> v1:172.21.15.138:6789/0 conn(0x7f8ca4164960 legacy=0x7f8ca40b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.632+0000 7f8ca37fe640 1 -- 172.21.15.94:0/3361829144 >> v1:172.21.15.94:6789/0 conn(0x7f8ca40b2eb0 legacy=0x7f8ca40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.632+0000 7f8ca27fc640 1 -- 172.21.15.94:0/3361829144 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 161042908 0 0) 0x7f8ca40b4c80 con 0x7f8ca4164960 2024-04-25T16:26:41.633 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.632+0000 7f8ca37fe640 1 -- 172.21.15.94:0/3361829144 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca40b4f60 con 0x7f8ca40d0120 2024-04-25T16:26:41.634 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.632+0000 7f8ca37fe640 1 --2- 172.21.15.94:0/3361829144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca4164420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097a0 tx=0x7f8c9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8ce5523014f644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:41.635 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.635+0000 7f8ca27fc640 1 -- 172.21.15.94:0/3361829144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98037070 con 0x7f8ca40d0120 2024-04-25T16:26:41.636 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.635+0000 7f8ca27fc640 1 -- 172.21.15.94:0/3361829144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c98032440 con 0x7f8ca40d0120 2024-04-25T16:26:41.636 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.635+0000 7f8ca27fc640 1 -- 172.21.15.94:0/3361829144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98032810 con 0x7f8ca40d0120 2024-04-25T16:26:41.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/3361829144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 msgr2=0x7f8ca4164420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/3361829144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca4164420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097a0 tx=0x7f8c9802a230 comp rx=0 tx=0).stop 2024-04-25T16:26:41.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/3361829144 shutdown_connections 2024-04-25T16:26:41.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/3361829144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca4164420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.637 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/3361829144 >> 172.21.15.94:0/3361829144 conn(0x7f8ca40b7ab0 msgr2=0x7f8ca40b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:41.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/3361829144 shutdown_connections 2024-04-25T16:26:41.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.636+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/3361829144 wait complete. 2024-04-25T16:26:41.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.638+0000 7f8ca8b9d640 1 Processor -- start 2024-04-25T16:26:41.638 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.638+0000 7f8ca8b9d640 1 -- start start 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca8b9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ca40b2eb0 0x7f8ca414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca8b9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca40c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca8b9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ca4164960 0x7f8ca415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca8b9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40b4920 con 0x7f8ca40d0120 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca8b9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40b1fa0 con 0x7f8ca40b2eb0 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca8b9d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca4161c40 con 0x7f8ca4164960 2024-04-25T16:26:41.641 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca40c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca40c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59012/0 (socket says 172.21.15.94:59012) 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 -- 172.21.15.94:0/2526131891 learned_addr learned my addr 172.21.15.94:0/2526131891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca37fe640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ca40b2eb0 0x7f8ca414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca3fff640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ca4164960 0x7f8ca415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 -- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ca4164960 msgr2=0x7f8ca415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ca4164960 0x7f8ca415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 -- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ca40b2eb0 msgr2=0x7f8ca414f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.639+0000 7f8ca2ffd640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ca40b2eb0 0x7f8ca414f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8ca2ffd640 1 -- 172.21.15.94:0/2526131891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca40c68c0 con 0x7f8ca40d0120 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8ca2ffd640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca40c80b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c440 tx=0x7f8c9400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94017070 con 0x7f8ca40d0120 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c940035f0 con 0x7f8ca40d0120 2024-04-25T16:26:41.642 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca40c6ba0 con 0x7f8ca40d0120 2024-04-25T16:26:41.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c940039e0 con 0x7f8ca40d0120 2024-04-25T16:26:41.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.640+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca40c7160 con 0x7f8ca40d0120 2024-04-25T16:26:41.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.641+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca40cf0f0 con 0x7f8ca40d0120 2024-04-25T16:26:41.643 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.641+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f8c9401a070 con 0x7f8ca40d0120 2024-04-25T16:26:41.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.642+0000 7f8c8bfff640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8c90074420 0x7f8c90076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:41.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.642+0000 7f8ca37fe640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8c90074420 0x7f8c90076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:41.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.642+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3351+0+0 (secure 0 0 0) 0x7f8c94093af0 con 0x7f8ca40d0120 2024-04-25T16:26:41.644 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.642+0000 7f8ca37fe640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8c90074420 0x7f8c90076930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8c98034000 tx=0x7f8c9802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:41.645 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:26:41.645+0000 7efda9f52640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T16:26:41.646 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:26:41.645+0000 7fb8108ff640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:41.647 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.647+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c9405e640 con 0x7f8ca40d0120 2024-04-25T16:26:41.657 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:26:41.656+0000 7efda1d4d640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:26:41.822 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.821+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ca4076ca0 con 0x7f8ca40d0120 2024-04-25T16:26:41.823 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.822+0000 7f8c8bfff640 1 -- 172.21.15.94:0/2526131891 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+8965 (secure 0 0 0) 0x7f8c94062940 con 0x7f8ca40d0120 2024-04-25T16:26:41.823 INFO:teuthology.misc.health.smithi094.stdout: 2024-04-25T16:26:41.823 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":13,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:26:41.475421+0000","last_up_change":"2024-04-25T16:26:41.475421+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8c90074420 msgr2=0x7f8c90076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8c90074420 0x7f8c90076930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8c98034000 tx=0x7f8c9802e000 comp rx=0 tx=0).stop 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 msgr2=0x7f8ca40c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca40c80b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c440 tx=0x7f8c9400c900 comp rx=0 tx=0).stop 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 shutdown_connections 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8c90074420 0x7f8c90076930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.825 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ca4164960 0x7f8ca415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.826 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d0120 0x7f8ca40c80b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.826 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 --2- 172.21.15.94:0/2526131891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ca40b2eb0 0x7f8ca414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:41.826 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 >> 172.21.15.94:0/2526131891 conn(0x7f8ca40b7ab0 msgr2=0x7f8ca4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:41.826 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 shutdown_connections 2024-04-25T16:26:41.826 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:41.824+0000 7f8ca8b9d640 1 -- 172.21.15.94:0/2526131891 wait complete. 2024-04-25T16:26:41.835 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-04-25T16:26:48.642 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T16:26:48.753 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.753+0000 7f3a9b8d9640 1 Processor -- start 2024-04-25T16:26:48.754 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.753+0000 7f3a9b8d9640 1 -- start start 2024-04-25T16:26:48.755 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a9b8d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3a940d0fc0 0x7f3a940d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a9b8d9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a940b33a0 0x7f3a940cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a9b8d9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a9415ee10 con 0x7f3a940b6de0 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a9b8d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3a940d0130 con 0x7f3a940d0fc0 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a9b8d9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a940717e0 con 0x7f3a940b33a0 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a98e74640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3a940d0fc0 0x7f3a940d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a98e74640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3a940d0fc0 0x7f3a940d2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59038/0 (socket says 172.21.15.94:59038) 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.754+0000 7f3a98e74640 1 -- 172.21.15.94:0/826909590 learned_addr learned my addr 172.21.15.94:0/826909590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.755+0000 7f3a99e76640 1 --2- 172.21.15.94:0/826909590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a940b33a0 0x7f3a940cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.755+0000 7f3a98e74640 1 -- 172.21.15.94:0/826909590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a940b33a0 msgr2=0x7f3a940cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.755+0000 7f3a98e74640 1 --2- 172.21.15.94:0/826909590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a940b33a0 0x7f3a940cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.755+0000 7f3a98e74640 1 -- 172.21.15.94:0/826909590 >> v1:172.21.15.138:6789/0 conn(0x7f3a940b6de0 legacy=0x7f3a940d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:48.756 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.755+0000 7f3a98e74640 1 -- 172.21.15.94:0/826909590 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a94071ac0 con 0x7f3a940d0fc0 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.755+0000 7f3a98e74640 1 --2- 172.21.15.94:0/826909590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940d0fc0 0x7f3a940d2130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e3c5d21a14fc13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a8bfff640 1 -- 172.21.15.94:0/826909590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84037070 con 0x7f3a940d0fc0 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a8bfff640 1 -- 172.21.15.94:0/826909590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a84032440 con 0x7f3a940d0fc0 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a8bfff640 1 -- 172.21.15.94:0/826909590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84032810 con 0x7f3a940d0fc0 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/826909590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940d0fc0 msgr2=0x7f3a940d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/826909590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940d0fc0 0x7f3a940d2130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8402a270 comp rx=0 tx=0).stop 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/826909590 shutdown_connections 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/826909590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a940b33a0 0x7f3a940cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.758 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/826909590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940d0fc0 0x7f3a940d2130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.759 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/826909590 >> 172.21.15.94:0/826909590 conn(0x7f3a940d3f90 msgr2=0x7f3a940d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:48.759 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/826909590 shutdown_connections 2024-04-25T16:26:48.759 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.756+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/826909590 wait complete. 2024-04-25T16:26:48.759 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.758+0000 7f3a9b8d9640 1 Processor -- start 2024-04-25T16:26:48.759 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.758+0000 7f3a9b8d9640 1 -- start start 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.759+0000 7f3a9b8d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 0x7f3a94153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a9b8d9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a940b6de0 0x7f3a940cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a9b8d9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a94153160 0x7f3a94153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a9b8d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a940cf120 con 0x7f3a940b33a0 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a9b8d9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a940b37b0 con 0x7f3a940b6de0 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a9b8d9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a940cf2a0 con 0x7f3a94153160 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 0x7f3a94153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 0x7f3a94153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59046/0 (socket says 172.21.15.94:59046) 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 -- 172.21.15.94:0/3435197127 learned_addr learned my addr 172.21.15.94:0/3435197127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99e76640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a94153160 0x7f3a94153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a98e74640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a940b6de0 0x7f3a940cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:48.762 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 -- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a94153160 msgr2=0x7f3a94153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a94153160 0x7f3a94153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 -- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a940b6de0 msgr2=0x7f3a940cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a940b6de0 0x7f3a940cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 -- 172.21.15.94:0/3435197127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a94160bc0 con 0x7f3a940b33a0 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99e76640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a94153160 0x7f3a94153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a99675640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 0x7f3a94153c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c730 tx=0x7f3a9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90016020 con 0x7f3a940b33a0 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a90006d50 con 0x7f3a940b33a0 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a940cb5a0 con 0x7f3a940b33a0 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.760+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90003bb0 con 0x7f3a940b33a0 2024-04-25T16:26:48.763 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.761+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a940cc1d0 con 0x7f3a940b33a0 2024-04-25T16:26:48.764 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.762+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a94050740 con 0x7f3a940b33a0 2024-04-25T16:26:48.764 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.762+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a9000a850 con 0x7f3a940b33a0 2024-04-25T16:26:48.764 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.763+0000 7f3a89ffb640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3a780744e0 0x7f3a780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:48.765 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.763+0000 7f3a98e74640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3a780744e0 0x7f3a780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:48.765 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.763+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3a90094b10 con 0x7f3a940b33a0 2024-04-25T16:26:48.765 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.763+0000 7f3a98e74640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3a780744e0 0x7f3a780769f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a84034000 tx=0x7f3a8402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:48.768 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.768+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a90020020 con 0x7f3a940b33a0 2024-04-25T16:26:48.945 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.945+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a940d3040 con 0x7f3a940b33a0 2024-04-25T16:26:48.946 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.946+0000 7f3a89ffb640 1 -- 172.21.15.94:0/3435197127 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12848 (secure 0 0 0) 0x7f3a9005f680 con 0x7f3a940b33a0 2024-04-25T16:26:48.946 INFO:teuthology.misc.health.smithi094.stdout: 2024-04-25T16:26:48.947 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":17,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:26:45.503245+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3a780744e0 msgr2=0x7f3a780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3a780744e0 0x7f3a780769f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a84034000 tx=0x7f3a8402e000 comp rx=0 tx=0).stop 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 msgr2=0x7f3a94153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 0x7f3a94153c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c730 tx=0x7f3a9000cbf0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 shutdown_connections 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3a780744e0 0x7f3a780769f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a94153160 0x7f3a94153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a940b6de0 0x7f3a940cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 --2- 172.21.15.94:0/3435197127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a940b33a0 0x7f3a94153c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 >> 172.21.15.94:0/3435197127 conn(0x7f3a940d3f90 msgr2=0x7f3a940d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:48.949 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 shutdown_connections 2024-04-25T16:26:48.950 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:26:48.948+0000 7f3a9b8d9640 1 -- 172.21.15.94:0/3435197127 wait complete. 2024-04-25T16:26:48.958 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T16:26:48.959 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T16:26:48.959 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T16:26:48.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T16:26:48.961 INFO:tasks.ceph.mds.a:Started 2024-04-25T16:26:48.961 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T16:26:48.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T16:26:48.963 INFO:tasks.ceph.mds.c:Started 2024-04-25T16:26:48.963 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T16:26:48.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T16:26:48.966 INFO:tasks.ceph.mds.b:Started 2024-04-25T16:26:48.966 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T16:26:48.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T16:26:48.968 INFO:tasks.ceph.mds.d:Started 2024-04-25T16:26:48.969 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T16:26:48.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:26:49.045 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-04-25T16:26:49.046 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-04-25T16:26:49.061 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-04-25T16:26:49.062 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-04-25T16:26:49.062 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-04-25T16:26:49.061+0000 7fa302be6640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6833/0: (98) Address already in use 2024-04-25T16:26:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.082+0000 7fded134d640 1 Processor -- start 2024-04-25T16:26:49.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.082+0000 7fded134d640 1 -- start start 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.083+0000 7fded134d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0d1720 0x7fdecc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fded134d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc06c9c0 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fded134d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc06cca0 con 0x7fdecc08e780 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fded134d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc06dfa0 con 0x7fdecc0d1720 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdeca7fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdecc0d4bb0 0x7fdecc0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46448/0 (socket says 172.21.15.94:46448) 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdeca7fc640 1 -- 172.21.15.94:0/3043065375 learned_addr learned my addr 172.21.15.94:0/3043065375 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdecb7fe640 1 --2- 172.21.15.94:0/3043065375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0d1720 0x7fdecc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548791642 0 0) 0x7fdecc06c9c0 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdeb4003520 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625692230 0 0) 0x7fdecc06cca0 con 0x7fdecc08e780 2024-04-25T16:26:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdecc06c9c0 con 0x7fdecc08e780 2024-04-25T16:26:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdecb7fe640 1 -- 172.21.15.94:0/3043065375 >> v1:172.21.15.138:6789/0 conn(0x7fdecc08e780 legacy=0x7fdecc0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156981415 0 0) 0x7fdeb4003520 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdecb7fe640 1 -- 172.21.15.94:0/3043065375 >> v1:172.21.15.94:6789/0 conn(0x7fdecc0d4bb0 legacy=0x7fdecc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.084+0000 7fdecb7fe640 1 -- 172.21.15.94:0/3043065375 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc06cf80 con 0x7fdecc0d1720 2024-04-25T16:26:49.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.085+0000 7fdecb7fe640 1 --2- 172.21.15.94:0/3043065375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0d1720 0x7fdecc06da60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009dc0 tx=0x7fdec402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d246c88c011904a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.085+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0001670 con 0x7fdecc0d1720 2024-04-25T16:26:49.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.085+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec4007540 con 0x7fdecc0d1720 2024-04-25T16:26:49.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.085+0000 7fdec9ffb640 1 -- 172.21.15.94:0/3043065375 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4007910 con 0x7fdecc0d1720 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 -- 172.21.15.94:0/3043065375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0d1720 msgr2=0x7fdecc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 --2- 172.21.15.94:0/3043065375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0d1720 0x7fdecc06da60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009dc0 tx=0x7fdec402a660 comp rx=0 tx=0).stop 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 -- 172.21.15.94:0/3043065375 shutdown_connections 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 --2- 172.21.15.94:0/3043065375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0d1720 0x7fdecc06da60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 -- 172.21.15.94:0/3043065375 >> 172.21.15.94:0/3043065375 conn(0x7fdecc0b7cc0 msgr2=0x7fdecc0b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 -- 172.21.15.94:0/3043065375 shutdown_connections 2024-04-25T16:26:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.086+0000 7fded134d640 1 -- 172.21.15.94:0/3043065375 wait complete. 2024-04-25T16:26:49.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.088+0000 7fded134d640 1 Processor -- start 2024-04-25T16:26:49.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.088+0000 7fded134d640 1 -- start start 2024-04-25T16:26:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fded134d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdecc08e780 0x7fdecc152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fded134d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 0x7fdecc151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fded134d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0ae8d0 0x7fdecc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fded134d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc06c570 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fded134d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc06d540 con 0x7fdecc08e780 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fded134d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc06c6f0 con 0x7fdecc0ae8d0 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 0x7fdecc151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 0x7fdecc151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59104/0 (socket says 172.21.15.94:59104) 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 -- 172.21.15.94:0/803171992 learned_addr learned my addr 172.21.15.94:0/803171992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdecb7fe640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0ae8d0 0x7fdecc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdecaffd640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdecc08e780 0x7fdecc152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 -- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0ae8d0 msgr2=0x7fdecc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0ae8d0 0x7fdecc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 -- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdecc08e780 msgr2=0x7fdecc152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdecc08e780 0x7fdecc152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 -- 172.21.15.94:0/803171992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc151b40 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.089+0000 7fdeca7fc640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 0x7fdecc151600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000fd0 tx=0x7fdec000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.090+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0017070 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.090+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0005ce0 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.090+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc0acca0 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.090+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00072b0 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.090+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc0ad260 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.091+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdecc06a180 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.091+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdec001a070 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.092+0000 7fdeb37fe640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb80744e0 0x7fdeb80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.092+0000 7fdecaffd640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb80744e0 0x7fdeb80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.092+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fdec0094130 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.092+0000 7fdecaffd640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb80744e0 0x7fdeb80769f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0011e0 tx=0x7fdebc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.096+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdec005e650 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.272+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdecc0d4620 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.272+0000 7fdeb37fe640 1 -- 172.21.15.94:0/803171992 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12848 (secure 0 0 0) 0x7fdec0062950 con 0x7fdecc0d4bb0 2024-04-25T16:26:49.273 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:26:49.273 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:26:45.503245+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:26:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb80744e0 msgr2=0x7fdeb80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb80744e0 0x7fdeb80769f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0011e0 tx=0x7fdebc009040 comp rx=0 tx=0).stop 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 msgr2=0x7fdecc151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 0x7fdecc151600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000fd0 tx=0x7fdec000c960 comp rx=0 tx=0).stop 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 shutdown_connections 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb80744e0 0x7fdeb80769f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdecc0ae8d0 0x7fdecc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdecc0d4bb0 0x7fdecc151600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 --2- 172.21.15.94:0/803171992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdecc08e780 0x7fdecc152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 >> 172.21.15.94:0/803171992 conn(0x7fdecc0b7cc0 msgr2=0x7fdecc0b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 shutdown_connections 2024-04-25T16:26:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.274+0000 7fded134d640 1 -- 172.21.15.94:0/803171992 wait complete. 2024-04-25T16:26:49.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:26:49.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:26:49.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.396+0000 7fa72c364640 1 Processor -- start 2024-04-25T16:26:49.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.396+0000 7fa72c364640 1 -- start start 2024-04-25T16:26:49.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72c364640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa7240b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72c364640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7240d18a0 0x7fa724161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72c364640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72406cdc0 con 0x7fa72406c570 2024-04-25T16:26:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72c364640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa72406d0a0 con 0x7fa7240d18a0 2024-04-25T16:26:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72c364640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa72406d380 con 0x7fa7241650b0 2024-04-25T16:26:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa7298ff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa72406c570 0x7fa7240d1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46452/0 (socket says 172.21.15.94:46452) 2024-04-25T16:26:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa7298ff640 1 -- 172.21.15.94:0/3295407497 learned_addr learned my addr 172.21.15.94:0/3295407497 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72a901640 1 --2- 172.21.15.94:0/3295407497 >> v2:172.21.15.138:3300/0 conn(0x7fa7240d18a0 0x7fa724161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa72a100640 1 --2- 172.21.15.94:0/3295407497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa7240b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa7290fe640 1 -- 172.21.15.94:0/3295407497 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979830749 0 0) 0x7fa72406cdc0 con 0x7fa72406c570 2024-04-25T16:26:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.398+0000 7fa7290fe640 1 -- 172.21.15.94:0/3295407497 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa710003520 con 0x7fa72406c570 2024-04-25T16:26:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa7290fe640 1 -- 172.21.15.94:0/3295407497 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 109293807 0 0) 0x7fa710003520 con 0x7fa72406c570 2024-04-25T16:26:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa72a100640 1 -- 172.21.15.94:0/3295407497 >> v2:172.21.15.138:3300/0 conn(0x7fa7240d18a0 msgr2=0x7fa724161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa72a100640 1 --2- 172.21.15.94:0/3295407497 >> v2:172.21.15.138:3300/0 conn(0x7fa7240d18a0 0x7fa724161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa72a100640 1 -- 172.21.15.94:0/3295407497 >> v1:172.21.15.94:6789/0 conn(0x7fa72406c570 legacy=0x7fa7240d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa72a100640 1 -- 172.21.15.94:0/3295407497 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72406d660 con 0x7fa7241650b0 2024-04-25T16:26:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa72a901640 1 --2- 172.21.15.94:0/3295407497 >> v2:172.21.15.138:3300/0 conn(0x7fa7240d18a0 0x7fa724161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.399+0000 7fa72a100640 1 --2- 172.21.15.94:0/3295407497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa7240b5460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d90 tx=0x7fa72002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b170b1f375a5ce16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa7290fe640 1 -- 172.21.15.94:0/3295407497 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714001670 con 0x7fa7241650b0 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa7290fe640 1 -- 172.21.15.94:0/3295407497 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa720007540 con 0x7fa7241650b0 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa7290fe640 1 -- 172.21.15.94:0/3295407497 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720007910 con 0x7fa7241650b0 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 -- 172.21.15.94:0/3295407497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 msgr2=0x7fa7240b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 --2- 172.21.15.94:0/3295407497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa7240b5460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d90 tx=0x7fa72002a560 comp rx=0 tx=0).stop 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 -- 172.21.15.94:0/3295407497 shutdown_connections 2024-04-25T16:26:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 --2- 172.21.15.94:0/3295407497 >> v2:172.21.15.138:3300/0 conn(0x7fa7240d18a0 0x7fa724161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 --2- 172.21.15.94:0/3295407497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa7240b5460 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 -- 172.21.15.94:0/3295407497 >> 172.21.15.94:0/3295407497 conn(0x7fa7240b7e90 msgr2=0x7fa7240b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 -- 172.21.15.94:0/3295407497 shutdown_connections 2024-04-25T16:26:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.400+0000 7fa72c364640 1 -- 172.21.15.94:0/3295407497 wait complete. 2024-04-25T16:26:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.402+0000 7fa72c364640 1 Processor -- start 2024-04-25T16:26:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.402+0000 7fa72c364640 1 -- start start 2024-04-25T16:26:49.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.403+0000 7fa72c364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 0x7fa72414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.403+0000 7fa72c364640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa72415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.403+0000 7fa72c364640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa724163900 0x7fa724163d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.403+0000 7fa72c364640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa72406c980 con 0x7fa72406c570 2024-04-25T16:26:49.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.403+0000 7fa72c364640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa72406de60 con 0x7fa724163900 2024-04-25T16:26:49.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.403+0000 7fa72c364640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa72406cb00 con 0x7fa7241650b0 2024-04-25T16:26:49.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 0x7fa72414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 0x7fa72414b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59114/0 (socket says 172.21.15.94:59114) 2024-04-25T16:26:49.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 -- 172.21.15.94:0/2722573389 learned_addr learned my addr 172.21.15.94:0/2722573389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:49.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa7298ff640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa72415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a901640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa724163900 0x7fa724163d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 -- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 msgr2=0x7fa72415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa72415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 -- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa724163900 msgr2=0x7fa724163d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa724163900 0x7fa724163d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 -- 172.21.15.94:0/2722573389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72415add0 con 0x7fa72406c570 2024-04-25T16:26:49.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72a100640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 0x7fa72414b690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa720009eb0 tx=0x7fa720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720040070 con 0x7fa72406c570 2024-04-25T16:26:49.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa720031c40 con 0x7fa72406c570 2024-04-25T16:26:49.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa724164250 con 0x7fa72406c570 2024-04-25T16:26:49.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720009450 con 0x7fa72406c570 2024-04-25T16:26:49.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.404+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa724157910 con 0x7fa72406c570 2024-04-25T16:26:49.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.405+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72406a180 con 0x7fa72406c570 2024-04-25T16:26:49.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.406+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa720039040 con 0x7fa72406c570 2024-04-25T16:26:49.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.406+0000 7fa71affd640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.406+0000 7fa7298ff640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.406+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa7200b6690 con 0x7fa72406c570 2024-04-25T16:26:49.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.407+0000 7fa7298ff640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa7241630d0 tx=0x7fa714009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.411+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa720081200 con 0x7fa72406c570 2024-04-25T16:26:49.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.589+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7240d4730 con 0x7fa72406c570 2024-04-25T16:26:49.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.589+0000 7fa71affd640 1 -- 172.21.15.94:0/2722573389 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fa720080b00 con 0x7fa72406c570 2024-04-25T16:26:49.590 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:26:49.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 msgr2=0x7fa7080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa7241630d0 tx=0x7fa714009040 comp rx=0 tx=0).stop 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 msgr2=0x7fa72414b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 0x7fa72414b690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa720009eb0 tx=0x7fa720004a40 comp rx=0 tx=0).stop 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 shutdown_connections 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa724163900 0x7fa724163d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7241650b0 0x7fa72415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 --2- 172.21.15.94:0/2722573389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa72406c570 0x7fa72414b690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.591+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 >> 172.21.15.94:0/2722573389 conn(0x7fa7240b7e90 msgr2=0x7fa7240b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.592+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 shutdown_connections 2024-04-25T16:26:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.592+0000 7fa72c364640 1 -- 172.21.15.94:0/2722573389 wait complete. 2024-04-25T16:26:49.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:26:49.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.709+0000 7f4ef911f640 1 Processor -- start 2024-04-25T16:26:49.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.709+0000 7f4ef911f640 1 -- start start 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef911f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ef4160840 0x7f4ef4164a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef911f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ef406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef911f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef4160c50 con 0x7f4ef40b4dd0 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef911f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef40b5670 con 0x7f4ef4160840 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef911f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef406c780 con 0x7f4ef406cc10 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef2d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4ef40b4dd0 0x7f4ef4164330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46460/0 (socket says 172.21.15.94:46460) 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef2d9d640 1 -- 172.21.15.94:0/3745086878 learned_addr learned my addr 172.21.15.94:0/3745086878 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.710+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187119041 0 0) 0x7f4ee4001110 con 0x7f4ef40b4dd0 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ee0003520 con 0x7f4ef40b4dd0 2024-04-25T16:26:49.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3745086878 >> v2:172.21.15.138:3300/0 conn(0x7f4ef4160840 0x7f4ef4164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef359e640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ef406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4024927000 0 0) 0x7f4ee0003520 con 0x7f4ef40b4dd0 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 msgr2=0x7f4ef406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ef406d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 >> v2:172.21.15.138:3300/0 conn(0x7f4ef4160840 msgr2=0x7f4ef4164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 --2- 172.21.15.94:0/3745086878 >> v2:172.21.15.138:3300/0 conn(0x7f4ef4160840 0x7f4ef4164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef40d57b0 con 0x7f4ef40b4dd0 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f4ee4001840 con 0x7f4ef40b4dd0 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 >> v1:172.21.15.94:6789/0 conn(0x7f4ef40b4dd0 legacy=0x7f4ef4164330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ee00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee0003f30 0x7f4ee0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0004d40 0x7f4ee0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef40b5670 con 0x7f4ee0003f30 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef406c780 con 0x7f4ee0004d40 2024-04-25T16:26:49.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee000bac0 con 0x7f4ef406cc10 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee0003f30 0x7f4ee0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef2d9d640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ee00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef359e640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0004d40 0x7f4ee0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 -- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 msgr2=0x7f4ee00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ee00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 -- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0004d40 msgr2=0x7f4ee0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0004d40 0x7f4ee0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4eea59c640 1 -- 172.21.15.94:0/3745086878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee0003520 con 0x7f4ee0003f30 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.711+0000 7f4ef2d9d640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef406cc10 0x7f4ee00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.712+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee0003f30 0x7f4ee0004800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009ac0 tx=0x7f4ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb61bec1cf09491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.712+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc036070 con 0x7f4ee0003f30 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.712+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc031400 con 0x7f4ee0003f30 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.712+0000 7f4ef259c640 1 -- 172.21.15.94:0/3745086878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc009690 con 0x7f4ee0003f30 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef911f640 1 -- 172.21.15.94:0/3745086878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee0003f30 msgr2=0x7f4ee0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3745086878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee0003f30 0x7f4ee0004800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009ac0 tx=0x7f4ee0005830 comp rx=0 tx=0).stop 2024-04-25T16:26:49.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef359e640 1 -- 172.21.15.94:0/3745086878 reap_dead start 2024-04-25T16:26:49.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef911f640 1 -- 172.21.15.94:0/3745086878 shutdown_connections 2024-04-25T16:26:49.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef911f640 1 -- 172.21.15.94:0/3745086878 >> 172.21.15.94:0/3745086878 conn(0x7f4ef40d6200 msgr2=0x7f4ef40d6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:49.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef911f640 1 -- 172.21.15.94:0/3745086878 shutdown_connections 2024-04-25T16:26:49.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.713+0000 7f4ef911f640 1 -- 172.21.15.94:0/3745086878 wait complete. 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.715+0000 7f4ef911f640 1 Processor -- start 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.715+0000 7f4ef911f640 1 -- start start 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef911f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0003f30 0x7f4ef41f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef911f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 0x7f4ef40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef911f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef40ca120 0x7f4ef40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef911f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4164f80 con 0x7f4ef40c9d10 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef911f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef40b5320 con 0x7f4ee0003f30 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef911f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef4165100 con 0x7f4ef40ca120 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4eea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 0x7f4ef40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4eea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 0x7f4ef40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59132/0 (socket says 172.21.15.94:59132) 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4eea59c640 1 -- 172.21.15.94:0/3935833147 learned_addr learned my addr 172.21.15.94:0/3935833147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef359e640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef40ca120 0x7f4ef40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.716+0000 7f4ef2d9d640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0003f30 0x7f4ef41f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eea59c640 1 -- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef40ca120 msgr2=0x7f4ef40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef40ca120 0x7f4ef40cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eea59c640 1 -- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0003f30 msgr2=0x7f4ef41f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0003f30 0x7f4ef41f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eea59c640 1 -- 172.21.15.94:0/3935833147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef40ca8f0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eea59c640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 0x7f4ef40cd240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4edc000c00 tx=0x7f4edc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc040020 con 0x7f4ef40c9d10 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc003c60 con 0x7f4ef40c9d10 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef40cabd0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc036040 con 0x7f4ef40c9d10 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.717+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef40cb190 con 0x7f4ef40c9d10 2024-04-25T16:26:49.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.718+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef40d41e0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.719+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4edc043020 con 0x7f4ef40c9d10 2024-04-25T16:26:49.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.719+0000 7f4eebfff640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4ed40744e0 0x7f4ed40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:49.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.719+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4edc0b53e0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.719+0000 7f4ef2d9d640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4ed40744e0 0x7f4ed40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:49.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.720+0000 7f4ef2d9d640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4ed40744e0 0x7f4ed40769f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4001eb0 tx=0x7f4ee4007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:49.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.724+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ef40d41e0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.936+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ef406deb0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.936+0000 7f4eebfff640 1 -- 172.21.15.94:0/3935833147 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f4ef406deb0 con 0x7f4ef40c9d10 2024-04-25T16:26:49.937 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:26:49.937 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T16:26:49.938 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-04-25T16:26:49.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4ed40744e0 msgr2=0x7f4ed40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4ed40744e0 0x7f4ed40769f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4001eb0 tx=0x7f4ee4007040 comp rx=0 tx=0).stop 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 msgr2=0x7f4ef40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 0x7f4ef40cd240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4edc000c00 tx=0x7f4edc004730 comp rx=0 tx=0).stop 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 shutdown_connections 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4ed40744e0 0x7f4ed40769f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ef40ca120 0x7f4ef40cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ef40c9d10 0x7f4ef40cd240 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 --2- 172.21.15.94:0/3935833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ee0003f30 0x7f4ef41f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.938+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 >> 172.21.15.94:0/3935833147 conn(0x7f4ef40d6200 msgr2=0x7f4ef40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.939+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 shutdown_connections 2024-04-25T16:26:49.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:49.939+0000 7f4ef911f640 1 -- 172.21.15.94:0/3935833147 wait complete. 2024-04-25T16:26:51.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:26:52.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:26:52.035 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:26:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.072+0000 7fb3b7d1f640 1 Processor -- start 2024-04-25T16:26:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.072+0000 7fb3b7d1f640 1 -- start start 2024-04-25T16:26:52.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.073+0000 7fb3b7d1f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b006da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b7d1f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3b00d4bb0 0x7fb3b00b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b7d1f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b00b5880 con 0x7fb3b00d1010 2024-04-25T16:26:52.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b7d1f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b006dfa0 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b7d1f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b006c9c0 con 0x7fb3b008e780 2024-04-25T16:26:52.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3b00d4bb0 0x7fb3b00b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b5abb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3b00d4bb0 0x7fb3b00b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52412/0 (socket says 172.21.15.94:52412) 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 -- 172.21.15.94:0/678146475 learned_addr learned my addr 172.21.15.94:0/678146475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 -- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 msgr2=0x7fb3b006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 --2- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b006da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 -- 172.21.15.94:0/678146475 >> v1:172.21.15.138:6789/0 conn(0x7fb3b00d1010 legacy=0x7fb3b00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b52ba640 1 -- 172.21.15.94:0/678146475 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b006cca0 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.074+0000 7fb3b5abb640 1 --2- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b006da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:52.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.075+0000 7fb3b52ba640 1 --2- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00b5340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009870 tx=0x7fb3a002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e1bbe1e465484c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.075+0000 7fb3b4ab9640 1 -- 172.21.15.94:0/678146475 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0037070 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.075+0000 7fb3b4ab9640 1 -- 172.21.15.94:0/678146475 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a0032440 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.075+0000 7fb3b4ab9640 1 -- 172.21.15.94:0/678146475 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0032810 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d4bb0 msgr2=0x7fb3b00b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00b5340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009870 tx=0x7fb3a002a270 comp rx=0 tx=0).stop 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/678146475 shutdown_connections 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00b5340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/678146475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b006da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/678146475 >> 172.21.15.94:0/678146475 conn(0x7fb3b00b7cc0 msgr2=0x7fb3b00b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:52.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/678146475 shutdown_connections 2024-04-25T16:26:52.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.076+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/678146475 wait complete. 2024-04-25T16:26:52.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.078+0000 7fb3b7d1f640 1 Processor -- start 2024-04-25T16:26:52.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.078+0000 7fb3b7d1f640 1 -- start start 2024-04-25T16:26:52.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b7d1f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b00cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b7d1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d1010 0x7fb3b00cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b7d1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b7d1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b006d540 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b7d1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b006d260 con 0x7fb3b00d1010 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b7d1f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b006c570 con 0x7fb3b008e780 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b62bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b62bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34836/0 (socket says 172.21.15.94:34836) 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b62bc640 1 -- 172.21.15.94:0/2265685698 learned_addr learned my addr 172.21.15.94:0/2265685698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b52ba640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d1010 0x7fb3b00cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.079+0000 7fb3b5abb640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b00cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b62bc640 1 -- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 msgr2=0x7fb3b00cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b62bc640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b00cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b62bc640 1 -- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d1010 msgr2=0x7fb3b00cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b62bc640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d1010 0x7fb3b00cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b62bc640 1 -- 172.21.15.94:0/2265685698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b01ea1f0 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b52ba640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d1010 0x7fb3b00cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b62bc640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00ce710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c6e0 tx=0x7fb3a800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003b70 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a8003d30 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8005310 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b01ea4d0 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.080+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01eaa90 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.081+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b00cfdc0 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.082+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb3a80054b0 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.082+0000 7fb3a67fc640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3940744b0 0x7fb3940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.082+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb3a8094c00 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.082+0000 7fb3b5abb640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3940744b0 0x7fb3940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.083+0000 7fb3b5abb640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3940744b0 0x7fb3940769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009560 tx=0x7fb3ac009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.087+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3a809b080 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.304+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3b006a180 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.305+0000 7fb3a67fc640 1 -- 172.21.15.94:0/2265685698 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7fb3a805f090 con 0x7fb3b00d4bb0 2024-04-25T16:26:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:26:52.306 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T16:26:52.306 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-04-25T16:26:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3940744b0 msgr2=0x7fb3940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3940744b0 0x7fb3940769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009560 tx=0x7fb3ac009190 comp rx=0 tx=0).stop 2024-04-25T16:26:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 msgr2=0x7fb3b00ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00ce710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c6e0 tx=0x7fb3a800cba0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 shutdown_connections 2024-04-25T16:26:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3940744b0 0x7fb3940769c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b00d4bb0 0x7fb3b00ce710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b00d1010 0x7fb3b00cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 --2- 172.21.15.94:0/2265685698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b008e780 0x7fb3b00cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 >> 172.21.15.94:0/2265685698 conn(0x7fb3b00b7cc0 msgr2=0x7fb3b00b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 shutdown_connections 2024-04-25T16:26:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.307+0000 7fb3b7d1f640 1 -- 172.21.15.94:0/2265685698 wait complete. 2024-04-25T16:26:52.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:26:52.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.466+0000 7f80bd372640 1 Processor -- start 2024-04-25T16:26:52.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.466+0000 7f80bd372640 1 -- start start 2024-04-25T16:26:52.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80bd372640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80b806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80bd372640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f80b80d1010 0x7f80b80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80bd372640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b80d1960 con 0x7f80b808e780 2024-04-25T16:26:52.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80bd372640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f80b806dfa0 con 0x7f80b80d1010 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80bd372640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80b806c9c0 con 0x7f80b80d4bb0 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b6ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f80b808e780 0x7f80b80b5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42016/0 (socket says 172.21.15.94:42016) 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b6ffd640 1 -- 172.21.15.94:0/2440042243 learned_addr learned my addr 172.21.15.94:0/2440042243 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b77fe640 1 --2- 172.21.15.94:0/2440042243 >> v2:172.21.15.138:3300/0 conn(0x7f80b80d1010 0x7f80b80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260080290 0 0) 0x7f80a8001110 con 0x7f80b808e780 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80affff640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80b806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a4003520 con 0x7f80b808e780 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 179523137 0 0) 0x7f80a4003520 con 0x7f80b808e780 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 msgr2=0x7f80b806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80b806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 >> v2:172.21.15.138:3300/0 conn(0x7f80b80d1010 msgr2=0x7f80b80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 --2- 172.21.15.94:0/2440042243 >> v2:172.21.15.138:3300/0 conn(0x7f80b80d1010 0x7f80b80d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b806cca0 con 0x7f80b808e780 2024-04-25T16:26:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f80a8001840 con 0x7f80b808e780 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 >> v1:172.21.15.94:6789/0 conn(0x7f80b808e780 legacy=0x7f80b80b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40039b0 0x7f80a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80a4004800 0x7f80a4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80a4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80b806dfa0 con 0x7f80a4004800 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80b806c9c0 con 0x7f80a40039b0 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.468+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80a400b9c0 con 0x7f80b80d4bb0 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80a4004800 0x7f80a4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80b6ffd640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40039b0 0x7f80a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80b77fe640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80a4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 -- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 msgr2=0x7f80a4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80a4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 -- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40039b0 msgr2=0x7f80a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40039b0 0x7f80a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 -- 172.21.15.94:0/2440042243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a4003520 con 0x7f80a4004800 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80b77fe640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b80d4bb0 0x7f80a4005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.469+0000 7f80affff640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80a4004800 0x7f80a4004c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ac0 tx=0x7f80a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60fb54fea55c530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0036070 con 0x7f80a4004800 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a0031400 con 0x7f80a4004800 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80b67fc640 1 -- 172.21.15.94:0/2440042243 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0009690 con 0x7f80a4004800 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80bd372640 1 -- 172.21.15.94:0/2440042243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80a4004800 msgr2=0x7f80a4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80bd372640 1 --2- 172.21.15.94:0/2440042243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80a4004800 0x7f80a4004c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ac0 tx=0x7f80a4005830 comp rx=0 tx=0).stop 2024-04-25T16:26:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80b77fe640 1 -- 172.21.15.94:0/2440042243 reap_dead start 2024-04-25T16:26:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80bd372640 1 -- 172.21.15.94:0/2440042243 shutdown_connections 2024-04-25T16:26:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80bd372640 1 -- 172.21.15.94:0/2440042243 >> 172.21.15.94:0/2440042243 conn(0x7f80b80b7cc0 msgr2=0x7f80b80b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80bd372640 1 -- 172.21.15.94:0/2440042243 shutdown_connections 2024-04-25T16:26:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.470+0000 7f80bd372640 1 -- 172.21.15.94:0/2440042243 wait complete. 2024-04-25T16:26:52.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.472+0000 7f80bd372640 1 Processor -- start 2024-04-25T16:26:52.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.472+0000 7f80bd372640 1 -- start start 2024-04-25T16:26:52.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80bd372640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a4004800 0x7f80b814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80bd372640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 0x7f80b815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80bd372640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b8157b80 0x7f80b815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80bd372640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80b806d540 con 0x7f80b81f7290 2024-04-25T16:26:52.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80bd372640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80b806d260 con 0x7f80a4004800 2024-04-25T16:26:52.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80bd372640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80b806c570 con 0x7f80b8157b80 2024-04-25T16:26:52.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 0x7f80b815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 0x7f80b815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34860/0 (socket says 172.21.15.94:34860) 2024-04-25T16:26:52.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.473+0000 7f80affff640 1 -- 172.21.15.94:0/239795101 learned_addr learned my addr 172.21.15.94:0/239795101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:52.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80b77fe640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b8157b80 0x7f80b815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80b6ffd640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a4004800 0x7f80b814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80affff640 1 -- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b8157b80 msgr2=0x7f80b815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80affff640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b8157b80 0x7f80b815f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80affff640 1 -- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a4004800 msgr2=0x7f80b814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80affff640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a4004800 0x7f80b814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80affff640 1 -- 172.21.15.94:0/239795101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b815c9b0 con 0x7f80b81f7290 2024-04-25T16:26:52.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80b77fe640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b8157b80 0x7f80b815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:52.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80b6ffd640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a4004800 0x7f80b814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:52.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80affff640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 0x7f80b815a800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ac0 tx=0x7f80a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0040020 con 0x7f80b81f7290 2024-04-25T16:26:52.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a0003bc0 con 0x7f80b81f7290 2024-04-25T16:26:52.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0036040 con 0x7f80b81f7290 2024-04-25T16:26:52.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b81f67b0 con 0x7f80b81f7290 2024-04-25T16:26:52.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.474+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81f6d70 con 0x7f80b81f7290 2024-04-25T16:26:52.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.475+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b815c720 con 0x7f80b81f7290 2024-04-25T16:26:52.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.476+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80a0043020 con 0x7f80b81f7290 2024-04-25T16:26:52.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.476+0000 7f80af7fe640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80980744b0 0x7f80980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.476+0000 7f80b6ffd640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80980744b0 0x7f80980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.476+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f80a00b5000 con 0x7f80b81f7290 2024-04-25T16:26:52.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.477+0000 7f80b6ffd640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80980744b0 0x7f80980769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80a80012f0 tx=0x7f80a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.481+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80a004a070 con 0x7f80b81f7290 2024-04-25T16:26:52.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.660+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80b806a180 con 0x7f80b81f7290 2024-04-25T16:26:52.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.661+0000 7f80af7fe640 1 -- 172.21.15.94:0/239795101 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12848 (secure 0 0 0) 0x7f80a00044e0 con 0x7f80b81f7290 2024-04-25T16:26:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:26:52.662 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:26:45.503245+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:26:52.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80980744b0 msgr2=0x7f80980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80980744b0 0x7f80980769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80a80012f0 tx=0x7f80a8009040 comp rx=0 tx=0).stop 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 msgr2=0x7f80b815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 0x7f80b815a800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ac0 tx=0x7f80a0004730 comp rx=0 tx=0).stop 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 shutdown_connections 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80980744b0 0x7f80980769c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80b8157b80 0x7f80b815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b81f7290 0x7f80b815a800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 --2- 172.21.15.94:0/239795101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a4004800 0x7f80b814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 >> 172.21.15.94:0/239795101 conn(0x7f80b80b7cc0 msgr2=0x7f80b80bad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 shutdown_connections 2024-04-25T16:26:52.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.663+0000 7f80bd372640 1 -- 172.21.15.94:0/239795101 wait complete. 2024-04-25T16:26:52.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:26:52.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:26:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.785+0000 7efedeff3640 1 Processor -- start 2024-04-25T16:26:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.785+0000 7efedeff3640 1 -- start start 2024-04-25T16:26:52.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efedeff3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efed80d3540 0x7efed81692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efedeff3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed80b2810 con 0x7efed806f590 2024-04-25T16:26:52.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efedeff3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed815f540 con 0x7efed80b2060 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efedeff3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efed815f820 con 0x7efed80d3540 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed7fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7efed806f590 0x7efed80d2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42028/0 (socket says 172.21.15.94:42028) 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed7fff640 1 -- 172.21.15.94:0/1363411141 learned_addr learned my addr 172.21.15.94:0/1363411141 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efedd590640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efed80d3540 0x7efed81692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868446736 0 0) 0x7efed80b2810 con 0x7efed806f590 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec0003520 con 0x7efed806f590 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584283228 0 0) 0x7efec0003520 con 0x7efed806f590 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efed80d3540 msgr2=0x7efed81692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efed80d3540 0x7efed81692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 >> v1:172.21.15.138:6789/0 conn(0x7efed80b2060 legacy=0x7efed815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed815fb00 con 0x7efed806f590 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7efecc001670 con 0x7efed806f590 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 >> v1:172.21.15.94:6789/0 conn(0x7efed806f590 legacy=0x7efed80d2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efec00039e0 0x7efec0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec0004800 0x7efec0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec0005150 0x7efec000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efed815f820 con 0x7efec00039e0 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.787+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efec000bf00 con 0x7efec0004800 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efec000c080 con 0x7efec0005150 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efec00039e0 0x7efec0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efed7fff640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec0004800 0x7efec0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedd590640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec0005150 0x7efec000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 -- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec0005150 msgr2=0x7efec000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec0005150 0x7efec000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 -- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec0004800 msgr2=0x7efec0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec0004800 0x7efec0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 -- 172.21.15.94:0/1363411141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec0003520 con 0x7efec00039e0 2024-04-25T16:26:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efed7fff640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec0004800 0x7efec0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.788+0000 7efedcd8f640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efec00039e0 0x7efec0003e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efec8001910 tx=0x7efec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63699d75c852b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8037070 con 0x7efec00039e0 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec802ac60 con 0x7efec00039e0 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efed77fe640 1 -- 172.21.15.94:0/1363411141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80014e0 con 0x7efec00039e0 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedeff3640 1 -- 172.21.15.94:0/1363411141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efec00039e0 msgr2=0x7efec0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedeff3640 1 --2- 172.21.15.94:0/1363411141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efec00039e0 0x7efec0003e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efec8001910 tx=0x7efec0005830 comp rx=0 tx=0).stop 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedd590640 1 -- 172.21.15.94:0/1363411141 reap_dead start 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedeff3640 1 -- 172.21.15.94:0/1363411141 shutdown_connections 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedeff3640 1 -- 172.21.15.94:0/1363411141 >> 172.21.15.94:0/1363411141 conn(0x7efed80b7cc0 msgr2=0x7efed80b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedeff3640 1 -- 172.21.15.94:0/1363411141 shutdown_connections 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.789+0000 7efedeff3640 1 -- 172.21.15.94:0/1363411141 wait complete. 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.791+0000 7efedeff3640 1 Processor -- start 2024-04-25T16:26:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.791+0000 7efedeff3640 1 -- start start 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.792+0000 7efedeff3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec00039e0 0x7efed815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.792+0000 7efedeff3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efed815a2e0 0x7efed81566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.792+0000 7efedeff3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 0x7efed815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedeff3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efed80d36c0 con 0x7efed8156c10 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedeff3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efed80d3540 con 0x7efed815a2e0 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedeff3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efed80b3fe0 con 0x7efec00039e0 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 0x7efed815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 0x7efed815c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34880/0 (socket says 172.21.15.94:34880) 2024-04-25T16:26:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 -- 172.21.15.94:0/3112223100 learned_addr learned my addr 172.21.15.94:0/3112223100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efed7fff640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efed815a2e0 0x7efed81566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 -- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec00039e0 msgr2=0x7efed815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec00039e0 0x7efed815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 -- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efed815a2e0 msgr2=0x7efed81566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efed815a2e0 0x7efed81566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 -- 172.21.15.94:0/3112223100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed8157020 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedd590640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 0x7efed815c580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efec0004780 tx=0x7efed00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0003990 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed001b070 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed8159520 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0003b70 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.793+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed8159ae0 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.794+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed806a180 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.795+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efed0005d10 con 0x7efed8156c10 2024-04-25T16:26:52.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.795+0000 7efed57fa640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efec40744b0 0x7efec40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:52.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.795+0000 7efedcd8f640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efec40744b0 0x7efec40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:52.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.795+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7efed0095350 con 0x7efed8156c10 2024-04-25T16:26:52.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.796+0000 7efedcd8f640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efec40744b0 0x7efec40769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efec8034000 tx=0x7efec802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:52.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.800+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efed8157020 con 0x7efed8156c10 2024-04-25T16:26:52.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.973+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efed8070bf0 con 0x7efed8156c10 2024-04-25T16:26:52.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.973+0000 7efed57fa640 1 -- 172.21.15.94:0/3112223100 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7efed005fe30 con 0x7efed8156c10 2024-04-25T16:26:52.974 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:26:52.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.975+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efec40744b0 msgr2=0x7efec40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.975+0000 7efedeff3640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efec40744b0 0x7efec40769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efec8034000 tx=0x7efec802c000 comp rx=0 tx=0).stop 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.975+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 msgr2=0x7efed815c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.975+0000 7efedeff3640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 0x7efed815c580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efec0004780 tx=0x7efed00115a0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 shutdown_connections 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efec40744b0 0x7efec40769c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed8156c10 0x7efed815c580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efed815a2e0 0x7efed81566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 --2- 172.21.15.94:0/3112223100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efec00039e0 0x7efed815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 >> 172.21.15.94:0/3112223100 conn(0x7efed80b7cc0 msgr2=0x7efed815ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 shutdown_connections 2024-04-25T16:26:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:52.976+0000 7efedeff3640 1 -- 172.21.15.94:0/3112223100 wait complete. 2024-04-25T16:26:52.988 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T16:26:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.094+0000 7f65ea6aa640 1 Processor -- start 2024-04-25T16:26:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.094+0000 7f65ea6aa640 1 -- start start 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.095+0000 7f65ea6aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f65e4160790 0x7f65e4164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.095+0000 7f65ea6aa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f65e406cea0 0x7f65e406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65ea6aa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e40b5a30 con 0x7f65e40b5230 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65ea6aa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f65e406ca10 con 0x7f65e4160790 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65ea6aa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f65e40d5610 con 0x7f65e406cea0 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f65e4160790 0x7f65e4164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f65e4160790 0x7f65e4164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34882/0 (socket says 172.21.15.94:34882) 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 -- 172.21.15.94:0/1404809544 learned_addr learned my addr 172.21.15.94:0/1404809544 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e8c47640 1 --2- 172.21.15.94:0/1404809544 >> v2:172.21.15.138:3300/0 conn(0x7f65e406cea0 0x7f65e406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 -- 172.21.15.94:0/1404809544 >> v2:172.21.15.138:3300/0 conn(0x7f65e406cea0 msgr2=0x7f65e406d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 --2- 172.21.15.94:0/1404809544 >> v2:172.21.15.138:3300/0 conn(0x7f65e406cea0 0x7f65e406d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 -- 172.21.15.94:0/1404809544 >> v1:172.21.15.138:6789/0 conn(0x7f65e40b5230 legacy=0x7f65e4164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:53.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.096+0000 7f65e37fe640 1 -- 172.21.15.94:0/1404809544 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e40d58f0 con 0x7f65e4160790 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.097+0000 7f65e37fe640 1 --2- 172.21.15.94:0/1404809544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e4164920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65e40b8da0 tx=0x7f65d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd8b2696fefed36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.097+0000 7f65e2ffd640 1 -- 172.21.15.94:0/1404809544 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8037070 con 0x7f65e4160790 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.097+0000 7f65e2ffd640 1 -- 172.21.15.94:0/1404809544 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d8032440 con 0x7f65e4160790 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.097+0000 7f65e2ffd640 1 -- 172.21.15.94:0/1404809544 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8032810 con 0x7f65e4160790 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.097+0000 7f65ea6aa640 1 -- 172.21.15.94:0/1404809544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 msgr2=0x7f65e4164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.097+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/1404809544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e4164920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65e40b8da0 tx=0x7f65d802a1a0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.098+0000 7f65ea6aa640 1 -- 172.21.15.94:0/1404809544 shutdown_connections 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.098+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/1404809544 >> v2:172.21.15.138:3300/0 conn(0x7f65e406cea0 0x7f65e406d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.098+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/1404809544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e4164920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.098+0000 7f65ea6aa640 1 -- 172.21.15.94:0/1404809544 >> 172.21.15.94:0/1404809544 conn(0x7f65e40d3570 msgr2=0x7f65e40b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.098+0000 7f65ea6aa640 1 -- 172.21.15.94:0/1404809544 shutdown_connections 2024-04-25T16:26:53.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.098+0000 7f65ea6aa640 1 -- 172.21.15.94:0/1404809544 wait complete. 2024-04-25T16:26:53.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.099+0000 7f65ea6aa640 1 Processor -- start 2024-04-25T16:26:53.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.099+0000 7f65ea6aa640 1 -- start start 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65ea6aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e406cea0 0x7f65e41f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65ea6aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e40b5230 0x7f65e40cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65ea6aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65ea6aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40b9870 con 0x7f65e4160790 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65ea6aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40b96f0 con 0x7f65e406cea0 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65ea6aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f65e4164e60 con 0x7f65e40b5230 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e40cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34890/0 (socket says 172.21.15.94:34890) 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 -- 172.21.15.94:0/3972762599 learned_addr learned my addr 172.21.15.94:0/3972762599 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e37fe640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e40b5230 0x7f65e40cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e3fff640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e406cea0 0x7f65e41f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 -- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e40b5230 msgr2=0x7f65e40cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e40b5230 0x7f65e40cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 -- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e406cea0 msgr2=0x7f65e41f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e406cea0 0x7f65e41f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 -- 172.21.15.94:0/3972762599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e40cc460 con 0x7f65e4160790 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e8c47640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e40cf710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cc30 tx=0x7f65dc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.101+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00ce40 con 0x7f65e4160790 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.102+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65dc003990 con 0x7f65e4160790 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.102+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e40cec30 con 0x7f65e4160790 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.102+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc003d40 con 0x7f65e4160790 2024-04-25T16:26:53.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.102+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e40cf1f0 con 0x7f65e4160790 2024-04-25T16:26:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.103+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e40cc1b0 con 0x7f65e4160790 2024-04-25T16:26:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.103+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f65dc003ee0 con 0x7f65e4160790 2024-04-25T16:26:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.104+0000 7f65e0ff9640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f65d00744e0 0x7f65d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.104+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f65dc095260 con 0x7f65e4160790 2024-04-25T16:26:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.104+0000 7f65e3fff640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f65d00744e0 0x7f65d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.104+0000 7f65e3fff640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f65d00744e0 0x7f65d00769f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f65d4006fd0 tx=0x7f65d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.108+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65dc014030 con 0x7f65e4160790 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.284+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f65e415fd80 con 0x7f65e4160790 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.284+0000 7f65e0ff9640 1 -- 172.21.15.94:0/3972762599 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f65dc05fdb0 con 0x7f65e4160790 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:26:53.285 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f65d00744e0 msgr2=0x7f65d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f65d00744e0 0x7f65d00769f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f65d4006fd0 tx=0x7f65d4008040 comp rx=0 tx=0).stop 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 msgr2=0x7f65e40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e40cf710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cc30 tx=0x7f65dc00a5c0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 shutdown_connections 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f65d00744e0 0x7f65d00769f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65e4160790 0x7f65e40cf710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e40b5230 0x7f65e40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 --2- 172.21.15.94:0/3972762599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e406cea0 0x7f65e41f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 >> 172.21.15.94:0/3972762599 conn(0x7f65e40d3570 msgr2=0x7f65e406e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 shutdown_connections 2024-04-25T16:26:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.286+0000 7f65ea6aa640 1 -- 172.21.15.94:0/3972762599 wait complete. 2024-04-25T16:26:53.298 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T16:26:53.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T16:26:53.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.408+0000 7fd4b8e92640 1 Processor -- start 2024-04-25T16:26:53.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.408+0000 7fd4b8e92640 1 -- start start 2024-04-25T16:26:53.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b8e92640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd4b40d4a80 0x7fd4b41642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b8e92640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4b41608b0 0x7fd4b41647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b8e92640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b40d4f80 con 0x7fd4b4164d30 2024-04-25T16:26:53.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b8e92640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b4165140 con 0x7fd4b40d4a80 2024-04-25T16:26:53.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b8e92640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b40d5260 con 0x7fd4b41608b0 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b259c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd4b40d4a80 0x7fd4b41642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b2d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd4b4164d30 0x7fd4b40d0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42032/0 (socket says 172.21.15.94:42032) 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b2d9d640 1 -- 172.21.15.94:0/915119981 learned_addr learned my addr 172.21.15.94:0/915119981 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b1d9b640 1 --2- 172.21.15.94:0/915119981 >> v2:172.21.15.138:3300/0 conn(0x7fd4b41608b0 0x7fd4b41647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b259c640 1 -- 172.21.15.94:0/915119981 >> v2:172.21.15.138:3300/0 conn(0x7fd4b41608b0 msgr2=0x7fd4b41647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b259c640 1 --2- 172.21.15.94:0/915119981 >> v2:172.21.15.138:3300/0 conn(0x7fd4b41608b0 0x7fd4b41647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b259c640 1 -- 172.21.15.94:0/915119981 >> v1:172.21.15.94:6789/0 conn(0x7fd4b4164d30 legacy=0x7fd4b40d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b259c640 1 -- 172.21.15.94:0/915119981 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b40d5540 con 0x7fd4b40d4a80 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.410+0000 7fd4b159a640 1 -- 172.21.15.94:0/915119981 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80220942 0 0) 0x7fd4b40d4f80 con 0x7fd4b4164d30 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.411+0000 7fd4b259c640 1 --2- 172.21.15.94:0/915119981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b40d4a80 0x7fd4b41642b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40b84e0 tx=0x7fd4a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca1b1d9c4c3cc4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.411+0000 7fd4b159a640 1 -- 172.21.15.94:0/915119981 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4037070 con 0x7fd4b40d4a80 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.411+0000 7fd4b159a640 1 -- 172.21.15.94:0/915119981 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a4032440 con 0x7fd4b40d4a80 2024-04-25T16:26:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.411+0000 7fd4b159a640 1 -- 172.21.15.94:0/915119981 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4032810 con 0x7fd4b40d4a80 2024-04-25T16:26:53.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.411+0000 7fd4b8e92640 1 -- 172.21.15.94:0/915119981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b40d4a80 msgr2=0x7fd4b41642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/915119981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b40d4a80 0x7fd4b41642b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40b84e0 tx=0x7fd4a402a1a0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 -- 172.21.15.94:0/915119981 shutdown_connections 2024-04-25T16:26:53.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/915119981 >> v2:172.21.15.138:3300/0 conn(0x7fd4b41608b0 0x7fd4b41647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/915119981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b40d4a80 0x7fd4b41642b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 -- 172.21.15.94:0/915119981 >> 172.21.15.94:0/915119981 conn(0x7fd4b40d6a00 msgr2=0x7fd4b40b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:53.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 -- 172.21.15.94:0/915119981 shutdown_connections 2024-04-25T16:26:53.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.412+0000 7fd4b8e92640 1 -- 172.21.15.94:0/915119981 wait complete. 2024-04-25T16:26:53.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.414+0000 7fd4b8e92640 1 Processor -- start 2024-04-25T16:26:53.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.414+0000 7fd4b8e92640 1 -- start start 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b8e92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b40d4a80 0x7fd4b4151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b8e92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41608b0 0x7fd4b4154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b8e92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 0x7fd4b4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b8e92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b40b5950 con 0x7fd4b4164d30 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b8e92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b40d5b00 con 0x7fd4b41608b0 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b8e92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b40b5ad0 con 0x7fd4b40d4a80 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 0x7fd4b4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 0x7fd4b4154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34902/0 (socket says 172.21.15.94:34902) 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 -- 172.21.15.94:0/3343601445 learned_addr learned my addr 172.21.15.94:0/3343601445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b259c640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b40d4a80 0x7fd4b4151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b1d9b640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41608b0 0x7fd4b4154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 -- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b40d4a80 msgr2=0x7fd4b4151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b40d4a80 0x7fd4b4151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 -- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41608b0 msgr2=0x7fd4b4154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41608b0 0x7fd4b4154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 -- 172.21.15.94:0/3343601445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b4153990 con 0x7fd4b4164d30 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.415+0000 7fd4b2d9d640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 0x7fd4b4154f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00cb20 tx=0x7fd4ac002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.416+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00abd0 con 0x7fd4b4164d30 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.416+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ac00ad90 con 0x7fd4b4164d30 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.416+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac012640 con 0x7fd4b4164d30 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.416+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b4153c70 con 0x7fd4b4164d30 2024-04-25T16:26:53.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.416+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b4154230 con 0x7fd4b4164d30 2024-04-25T16:26:53.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.417+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b406a180 con 0x7fd4b4164d30 2024-04-25T16:26:53.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.417+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4ac0127e0 con 0x7fd4b4164d30 2024-04-25T16:26:53.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.418+0000 7fd4a2ffd640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd4940744b0 0x7fd4940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:53.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.418+0000 7fd4b259c640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd4940744b0 0x7fd4940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:53.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.418+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd4ac0953b0 con 0x7fd4b4164d30 2024-04-25T16:26:53.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.418+0000 7fd4b259c640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd4940744b0 0x7fd4940769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4034000 tx=0x7fd4a402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:53.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.422+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4ac05f840 con 0x7fd4b4164d30 2024-04-25T16:26:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:53.599+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd4b40d4630 con 0x7fd4b4164d30 2024-04-25T16:26:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.529+0000 7fd4a2ffd640 1 -- 172.21.15.94:0/3343601445 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fd4ac063b40 con 0x7fd4b4164d30 2024-04-25T16:26:54.530 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-04-25T16:26:54.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.531+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd4940744b0 msgr2=0x7fd4940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.531+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd4940744b0 0x7fd4940769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4034000 tx=0x7fd4a402e000 comp rx=0 tx=0).stop 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.531+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 msgr2=0x7fd4b4154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.531+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 0x7fd4b4154f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00cb20 tx=0x7fd4ac002190 comp rx=0 tx=0).stop 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 shutdown_connections 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd4940744b0 0x7fd4940769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b4164d30 0x7fd4b4154f70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41608b0 0x7fd4b4154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 --2- 172.21.15.94:0/3343601445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b40d4a80 0x7fd4b4151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 >> 172.21.15.94:0/3343601445 conn(0x7fd4b40d6a00 msgr2=0x7fd4b40b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 shutdown_connections 2024-04-25T16:26:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.532+0000 7fd4b8e92640 1 -- 172.21.15.94:0/3343601445 wait complete. 2024-04-25T16:26:54.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T16:26:54.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.667+0000 7fc092157640 1 Processor -- start 2024-04-25T16:26:54.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.667+0000 7fc092157640 1 -- start start 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc092157640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc08c0b66a0 0x7fc08c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc092157640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc092157640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc08c08d950 con 0x7fc08c0cf4d0 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc092157640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc08c0b9270 con 0x7fc08c0b66a0 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc092157640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc08c08dc30 con 0x7fc08c0ceb80 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08b7fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc08c0b66a0 0x7fc08c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08bfff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc08c0cf4d0 0x7fc08c0d1d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:42788/0 (socket says 172.21.15.94:42788) 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:54.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08bfff640 1 -- 172.21.15.94:0/1526680563 learned_addr learned my addr 172.21.15.94:0/1526680563 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08affd640 1 -- 172.21.15.94:0/1526680563 >> v2:172.21.15.138:3300/0 conn(0x7fc08c0b66a0 msgr2=0x7fc08c0b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08affd640 1 --2- 172.21.15.94:0/1526680563 >> v2:172.21.15.138:3300/0 conn(0x7fc08c0b66a0 0x7fc08c0b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.669+0000 7fc08affd640 1 -- 172.21.15.94:0/1526680563 >> v1:172.21.15.138:6789/0 conn(0x7fc08c0cf4d0 legacy=0x7fc08c0d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.670+0000 7fc08a7fc640 1 -- 172.21.15.94:0/1526680563 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910853968 0 0) 0x7fc08c08d950 con 0x7fc08c0cf4d0 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.670+0000 7fc08affd640 1 -- 172.21.15.94:0/1526680563 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c08df10 con 0x7fc08c0ceb80 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.670+0000 7fc08b7fe640 1 --2- 172.21.15.94:0/1526680563 >> v2:172.21.15.138:3300/0 conn(0x7fc08c0b66a0 0x7fc08c0b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.670+0000 7fc08affd640 1 --2- 172.21.15.94:0/1526680563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c0cef90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097a0 tx=0x7fc08002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c6868e7a419ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.670+0000 7fc08a7fc640 1 -- 172.21.15.94:0/1526680563 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080037070 con 0x7fc08c0ceb80 2024-04-25T16:26:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc08a7fc640 1 -- 172.21.15.94:0/1526680563 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc080032440 con 0x7fc08c0ceb80 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc08a7fc640 1 -- 172.21.15.94:0/1526680563 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080032810 con 0x7fc08c0ceb80 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 -- 172.21.15.94:0/1526680563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 msgr2=0x7fc08c0cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 --2- 172.21.15.94:0/1526680563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c0cef90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097a0 tx=0x7fc08002a180 comp rx=0 tx=0).stop 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 -- 172.21.15.94:0/1526680563 shutdown_connections 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 --2- 172.21.15.94:0/1526680563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c0cef90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 --2- 172.21.15.94:0/1526680563 >> v2:172.21.15.138:3300/0 conn(0x7fc08c0b66a0 0x7fc08c0b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 -- 172.21.15.94:0/1526680563 >> 172.21.15.94:0/1526680563 conn(0x7fc08c0b5a60 msgr2=0x7fc08c0b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 -- 172.21.15.94:0/1526680563 shutdown_connections 2024-04-25T16:26:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.671+0000 7fc092157640 1 -- 172.21.15.94:0/1526680563 wait complete. 2024-04-25T16:26:54.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.673+0000 7fc092157640 1 Processor -- start 2024-04-25T16:26:54.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.673+0000 7fc092157640 1 -- start start 2024-04-25T16:26:54.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.674+0000 7fc092157640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 0x7fc08c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:54.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.674+0000 7fc092157640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c1f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc092157640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c0cf4d0 0x7fc08c0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc092157640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c0cfa60 con 0x7fc08c0b66a0 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc092157640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c0cf8e0 con 0x7fc08c0cf4d0 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc092157640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc08c0b7f80 con 0x7fc08c0ceb80 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 0x7fc08c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 0x7fc08c15e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34910/0 (socket says 172.21.15.94:34910) 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 -- 172.21.15.94:0/918465350 learned_addr learned my addr 172.21.15.94:0/918465350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08bfff640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c0cf4d0 0x7fc08c0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08affd640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c1f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 -- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 msgr2=0x7fc08c1f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c1f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 -- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c0cf4d0 msgr2=0x7fc08c0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c0cf4d0 0x7fc08c0cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:54.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 -- 172.21.15.94:0/918465350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1f4390 con 0x7fc08c0b66a0 2024-04-25T16:26:54.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc08b7fe640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 0x7fc08c15e330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e910 tx=0x7fc07c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:54.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c017070 con 0x7fc08c0b66a0 2024-04-25T16:26:54.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07c00ca00 con 0x7fc08c0b66a0 2024-04-25T16:26:54.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c15eaf0 con 0x7fc08c0b66a0 2024-04-25T16:26:54.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cdb0 con 0x7fc08c0b66a0 2024-04-25T16:26:54.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.675+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c15f0b0 con 0x7fc08c0b66a0 2024-04-25T16:26:54.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.676+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c06a180 con 0x7fc08c0b66a0 2024-04-25T16:26:54.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.677+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc07c01a070 con 0x7fc08c0b66a0 2024-04-25T16:26:54.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.677+0000 7fc073fff640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0780744b0 0x7fc0780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:54.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.678+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fc07c0952d0 con 0x7fc08c0b66a0 2024-04-25T16:26:54.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.678+0000 7fc08affd640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0780744b0 0x7fc0780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:54.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.678+0000 7fc08affd640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0780744b0 0x7fc0780769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc080034000 tx=0x7fc08002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:54.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.682+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc07c05fc50 con 0x7fc08c0b66a0 2024-04-25T16:26:54.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:54.856+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc08c073530 con 0x7fc08c0b66a0 2024-04-25T16:26:55.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.533+0000 7fc073fff640 1 -- 172.21.15.94:0/918465350 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fc07c05f550 con 0x7fc08c0b66a0 2024-04-25T16:26:55.534 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0780744b0 msgr2=0x7fc0780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0780744b0 0x7fc0780769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc080034000 tx=0x7fc08002e000 comp rx=0 tx=0).stop 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 msgr2=0x7fc08c15e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 0x7fc08c15e330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e910 tx=0x7fc07c00edd0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 shutdown_connections 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0780744b0 0x7fc0780769c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c0cf4d0 0x7fc08c0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc08c0ceb80 0x7fc08c1f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 --2- 172.21.15.94:0/918465350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c0b66a0 0x7fc08c15e330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 >> 172.21.15.94:0/918465350 conn(0x7fc08c0b5a60 msgr2=0x7fc08c0c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:55.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 shutdown_connections 2024-04-25T16:26:55.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.537+0000 7fc092157640 1 -- 172.21.15.94:0/918465350 wait complete. 2024-04-25T16:26:55.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T16:26:55.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.670+0000 7fca310d8640 1 Processor -- start 2024-04-25T16:26:55.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.670+0000 7fca310d8640 1 -- start start 2024-04-25T16:26:55.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca310d8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fca2c0b5710 0x7fca2c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca310d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c160e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca310d8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca2c161360 0x7fca2c0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca310d8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c161c10 con 0x7fca2c0b5710 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca310d8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c161930 con 0x7fca2c161360 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca310d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c164d10 con 0x7fca2c0b4c70 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fca2c0b5710 0x7fca2c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fca2c0b5710 0x7fca2c1608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34912/0 (socket says 172.21.15.94:34912) 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 -- 172.21.15.94:0/3761554325 learned_addr learned my addr 172.21.15.94:0/3761554325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2a59c640 1 --2- 172.21.15.94:0/3761554325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2b59e640 1 --2- 172.21.15.94:0/3761554325 >> v2:172.21.15.138:3300/0 conn(0x7fca2c161360 0x7fca2c0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 -- 172.21.15.94:0/3761554325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 msgr2=0x7fca2c160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 --2- 172.21.15.94:0/3761554325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 -- 172.21.15.94:0/3761554325 >> v2:172.21.15.138:3300/0 conn(0x7fca2c161360 msgr2=0x7fca2c0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:55.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 --2- 172.21.15.94:0/3761554325 >> v2:172.21.15.138:3300/0 conn(0x7fca2c161360 0x7fca2c0d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.672+0000 7fca2ad9d640 1 -- 172.21.15.94:0/3761554325 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c164ff0 con 0x7fca2c0b5710 2024-04-25T16:26:55.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.673+0000 7fca2ad9d640 1 --2- 172.21.15.94:0/3761554325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1608e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0d2720 tx=0x7fca1c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bb2ffa85ac6b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:55.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.673+0000 7fca29d9b640 1 -- 172.21.15.94:0/3761554325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c036070 con 0x7fca2c0b5710 2024-04-25T16:26:55.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.673+0000 7fca29d9b640 1 -- 172.21.15.94:0/3761554325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c031600 con 0x7fca2c0b5710 2024-04-25T16:26:55.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.673+0000 7fca29d9b640 1 -- 172.21.15.94:0/3761554325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0319d0 con 0x7fca2c0b5710 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 -- 172.21.15.94:0/3761554325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 msgr2=0x7fca2c1608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 --2- 172.21.15.94:0/3761554325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1608e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0d2720 tx=0x7fca1c02a570 comp rx=0 tx=0).stop 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 -- 172.21.15.94:0/3761554325 shutdown_connections 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 --2- 172.21.15.94:0/3761554325 >> v2:172.21.15.138:3300/0 conn(0x7fca2c161360 0x7fca2c0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 --2- 172.21.15.94:0/3761554325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 --2- 172.21.15.94:0/3761554325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1608e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 -- 172.21.15.94:0/3761554325 >> 172.21.15.94:0/3761554325 conn(0x7fca2c0d6930 msgr2=0x7fca2c0b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 -- 172.21.15.94:0/3761554325 shutdown_connections 2024-04-25T16:26:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.674+0000 7fca310d8640 1 -- 172.21.15.94:0/3761554325 wait complete. 2024-04-25T16:26:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.676+0000 7fca310d8640 1 Processor -- start 2024-04-25T16:26:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.676+0000 7fca310d8640 1 -- start start 2024-04-25T16:26:55.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca310d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca310d8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca310d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c161360 0x7fca2c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca310d8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c0d1260 con 0x7fca2c0b5710 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca310d8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c0b51c0 con 0x7fca2c161360 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca310d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c0d13e0 con 0x7fca2c0b4c70 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca2a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca2a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34914/0 (socket says 172.21.15.94:34914) 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca2a59c640 1 -- 172.21.15.94:0/2924155260 learned_addr learned my addr 172.21.15.94:0/2924155260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca2ad9d640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.677+0000 7fca2b59e640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c161360 0x7fca2c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca2a59c640 1 -- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 msgr2=0x7fca2c152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca2a59c640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca2a59c640 1 -- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c161360 msgr2=0x7fca2c0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca2a59c640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c161360 0x7fca2c0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca2a59c640 1 -- 172.21.15.94:0/2924155260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c152c40 con 0x7fca2c0b5710 2024-04-25T16:26:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca2a59c640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1f72d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d800 tx=0x7fca2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20018030 con 0x7fca2c0b5710 2024-04-25T16:26:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca200096a0 con 0x7fca2c0b5710 2024-04-25T16:26:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c1f7810 con 0x7fca2c0b5710 2024-04-25T16:26:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20009870 con 0x7fca2c0b5710 2024-04-25T16:26:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.678+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c0cb810 con 0x7fca2c0b5710 2024-04-25T16:26:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.679+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2c06a180 con 0x7fca2c0b5710 2024-04-25T16:26:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.680+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fca2001b070 con 0x7fca2c0b5710 2024-04-25T16:26:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.680+0000 7fca137fe640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fca180744b0 0x7fca180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.680+0000 7fca2ad9d640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fca180744b0 0x7fca180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.680+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fca20095340 con 0x7fca2c0b5710 2024-04-25T16:26:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.680+0000 7fca2ad9d640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fca180744b0 0x7fca180769c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca1c033000 tx=0x7fca1c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:55.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.685+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca2c152c40 con 0x7fca2c0b5710 2024-04-25T16:26:55.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:55.898+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fca2c06f940 con 0x7fca2c0b5710 2024-04-25T16:26:56.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.535+0000 7fca137fe640 1 -- 172.21.15.94:0/2924155260 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fca2005f4e0 con 0x7fca2c0b5710 2024-04-25T16:26:56.536 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T16:26:56.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.538+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fca180744b0 msgr2=0x7fca180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:56.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.538+0000 7fca310d8640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fca180744b0 0x7fca180769c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca1c033000 tx=0x7fca1c009000 comp rx=0 tx=0).stop 2024-04-25T16:26:56.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.538+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 msgr2=0x7fca2c1f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:56.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.538+0000 7fca310d8640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1f72d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d800 tx=0x7fca2000dcc0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.540+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 shutdown_connections 2024-04-25T16:26:56.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.540+0000 7fca310d8640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fca180744b0 0x7fca180769c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.540+0000 7fca310d8640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c161360 0x7fca2c0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.540+0000 7fca310d8640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca2c0b5710 0x7fca2c1f72d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.540+0000 7fca310d8640 1 --2- 172.21.15.94:0/2924155260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c0b4c70 0x7fca2c152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.540+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 >> 172.21.15.94:0/2924155260 conn(0x7fca2c0d6930 msgr2=0x7fca2c0b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.545+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 shutdown_connections 2024-04-25T16:26:56.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.545+0000 7fca310d8640 1 -- 172.21.15.94:0/2924155260 wait complete. 2024-04-25T16:26:56.573 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T16:26:56.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T16:26:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.685+0000 7fa096c14640 1 Processor -- start 2024-04-25T16:26:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.685+0000 7fa096c14640 1 -- start start 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa096c14640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 0x7fa0901649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa096c14640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa09006c7e0 0x7fa09006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa096c14640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0900d4f80 con 0x7fa0900d4a80 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa096c14640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa0900d1e00 con 0x7fa09006c7e0 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa096c14640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa0900d5260 con 0x7fa0901608b0 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa0949b0640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa0900d4a80 0x7fa0901642b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42040/0 (socket says 172.21.15.94:42040) 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.686+0000 7fa0949b0640 1 -- 172.21.15.94:0/3597824110 learned_addr learned my addr 172.21.15.94:0/3597824110 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1044983988 0 0) 0x7fa0900d4f80 con 0x7fa0900d4a80 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa0951b1640 1 --2- 172.21.15.94:0/3597824110 >> v2:172.21.15.138:3300/0 conn(0x7fa09006c7e0 0x7fa09006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 0x7fa0901649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa080003520 con 0x7fa0900d4a80 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2150079267 0 0) 0x7fa080003520 con 0x7fa0900d4a80 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 msgr2=0x7fa0901649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 0x7fa0901649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 >> v2:172.21.15.138:3300/0 conn(0x7fa09006c7e0 msgr2=0x7fa09006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 --2- 172.21.15.94:0/3597824110 >> v2:172.21.15.138:3300/0 conn(0x7fa09006c7e0 0x7fa09006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0900d5540 con 0x7fa0900d4a80 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fa084001670 con 0x7fa0900d4a80 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 >> v1:172.21.15.94:6789/0 conn(0x7fa0900d4a80 legacy=0x7fa0901642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0800039b0 0x7fa080003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa080004800 0x7fa080004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 0x7fa080005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900d1e00 con 0x7fa080004800 2024-04-25T16:26:56.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900d5260 con 0x7fa0800039b0 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa08000b9c0 con 0x7fa0901608b0 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa080004800 0x7fa080004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa0949b0640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0800039b0 0x7fa080003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa0951b1640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 0x7fa080005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 -- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 msgr2=0x7fa080005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0901608b0 0x7fa080005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 -- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0800039b0 msgr2=0x7fa080003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0800039b0 0x7fa080003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.687+0000 7fa08ffff640 1 -- 172.21.15.94:0/3597824110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa080003520 con 0x7fa080004800 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.688+0000 7fa08ffff640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa080004800 0x7fa080004c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009ac0 tx=0x7fa080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c128bf17b03d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.688+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c036070 con 0x7fa080004800 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.688+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c031400 con 0x7fa080004800 2024-04-25T16:26:56.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.688+0000 7fa08f7fe640 1 -- 172.21.15.94:0/3597824110 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c009690 con 0x7fa080004800 2024-04-25T16:26:56.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa096c14640 1 -- 172.21.15.94:0/3597824110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa080004800 msgr2=0x7fa080004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:56.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa096c14640 1 --2- 172.21.15.94:0/3597824110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa080004800 0x7fa080004c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009ac0 tx=0x7fa080005830 comp rx=0 tx=0).stop 2024-04-25T16:26:56.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa0951b1640 1 -- 172.21.15.94:0/3597824110 reap_dead start 2024-04-25T16:26:56.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa096c14640 1 -- 172.21.15.94:0/3597824110 shutdown_connections 2024-04-25T16:26:56.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa096c14640 1 -- 172.21.15.94:0/3597824110 >> 172.21.15.94:0/3597824110 conn(0x7fa0900d6a00 msgr2=0x7fa0900b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:56.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa096c14640 1 -- 172.21.15.94:0/3597824110 shutdown_connections 2024-04-25T16:26:56.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.689+0000 7fa096c14640 1 -- 172.21.15.94:0/3597824110 wait complete. 2024-04-25T16:26:56.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.691+0000 7fa096c14640 1 Processor -- start 2024-04-25T16:26:56.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.691+0000 7fa096c14640 1 -- start start 2024-04-25T16:26:56.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa096c14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080004800 0x7fa090160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa096c14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 0x7fa09015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa096c14640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0900b6f40 0x7fa0900b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa096c14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900b5950 con 0x7fa09014d040 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa096c14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900d5b00 con 0x7fa080004800 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa096c14640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa0900b5ad0 con 0x7fa0900b6f40 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 0x7fa09015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 0x7fa09015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34926/0 (socket says 172.21.15.94:34926) 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa08ffff640 1 -- 172.21.15.94:0/1606153517 learned_addr learned my addr 172.21.15.94:0/1606153517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa0949b0640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080004800 0x7fa090160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.692+0000 7fa0951b1640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0900b6f40 0x7fa0900b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08ffff640 1 -- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0900b6f40 msgr2=0x7fa0900b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08ffff640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0900b6f40 0x7fa0900b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08ffff640 1 -- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080004800 msgr2=0x7fa090160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08ffff640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080004800 0x7fa090160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08ffff640 1 -- 172.21.15.94:0/1606153517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0900b7890 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa0951b1640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0900b6f40 0x7fa0900b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08ffff640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 0x7fa09015c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009ac0 tx=0x7fa07c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c040020 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c003bc0 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c036040 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0901615c0 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.693+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa090161b80 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.694+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa090161330 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.695+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa07c043020 con 0x7fa09014d040 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.695+0000 7fa08d7fa640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa0740744b0 0x7fa0740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.695+0000 7fa0949b0640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa0740744b0 0x7fa0740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:56.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.695+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa07c0b52d0 con 0x7fa09014d040 2024-04-25T16:26:56.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.695+0000 7fa0949b0640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa0740744b0 0x7fa0740769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa084001eb0 tx=0x7fa084007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:56.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.700+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa07c04a070 con 0x7fa09014d040 2024-04-25T16:26:56.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:56.885+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa09006a180 con 0x7fa09014d040 2024-04-25T16:26:57.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.536+0000 7fa08d7fa640 1 -- 172.21.15.94:0/1606153517 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fa0900b7890 con 0x7fa09014d040 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa0740744b0 msgr2=0x7fa0740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa0740744b0 0x7fa0740769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa084001eb0 tx=0x7fa084007970 comp rx=0 tx=0).stop 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 msgr2=0x7fa09015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 0x7fa09015c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009ac0 tx=0x7fa07c004730 comp rx=0 tx=0).stop 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 shutdown_connections 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa0740744b0 0x7fa0740769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0900b6f40 0x7fa0900b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa09014d040 0x7fa09015c2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 --2- 172.21.15.94:0/1606153517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080004800 0x7fa090160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 >> 172.21.15.94:0/1606153517 conn(0x7fa0900d6a00 msgr2=0x7fa0900b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 shutdown_connections 2024-04-25T16:26:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.539+0000 7fa096c14640 1 -- 172.21.15.94:0/1606153517 wait complete. 2024-04-25T16:26:57.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T16:26:57.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.667+0000 7f058554d640 1 Processor -- start 2024-04-25T16:26:57.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.667+0000 7f058554d640 1 -- start start 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f058554d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f05800b56a0 0x7f05800b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f058554d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05800b9cc0 0x7f05800d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f058554d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058006c8c0 con 0x7f05800d4aa0 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f058554d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f058006cba0 con 0x7f05800b56a0 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f058554d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f058006ce80 con 0x7f05800b9cc0 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f05800b56a0 0x7f05800b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f05800b56a0 0x7f05800b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34936/0 (socket says 172.21.15.94:34936) 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 -- 172.21.15.94:0/775066205 learned_addr learned my addr 172.21.15.94:0/775066205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057f7fe640 1 --2- 172.21.15.94:0/775066205 >> v2:172.21.15.138:3300/0 conn(0x7f05800b9cc0 0x7f05800d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:57.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 -- 172.21.15.94:0/775066205 >> v2:172.21.15.138:3300/0 conn(0x7f05800b9cc0 msgr2=0x7f05800d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:57.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 --2- 172.21.15.94:0/775066205 >> v2:172.21.15.138:3300/0 conn(0x7f05800b9cc0 0x7f05800d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 -- 172.21.15.94:0/775066205 >> v1:172.21.15.138:6789/0 conn(0x7f05800d4aa0 legacy=0x7f058006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:57.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.669+0000 7f057e7fc640 1 -- 172.21.15.94:0/775066205 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058006d160 con 0x7f05800b56a0 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.670+0000 7f057e7fc640 1 --2- 172.21.15.94:0/775066205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b56a0 0x7f05800b9780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05800d40b0 tx=0x7f057402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f959c00b01e80f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.670+0000 7f057dffb640 1 -- 172.21.15.94:0/775066205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574037070 con 0x7f05800b56a0 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.670+0000 7f057dffb640 1 -- 172.21.15.94:0/775066205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574032440 con 0x7f05800b56a0 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.670+0000 7f057dffb640 1 -- 172.21.15.94:0/775066205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574032810 con 0x7f05800b56a0 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 -- 172.21.15.94:0/775066205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b56a0 msgr2=0x7f05800b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 --2- 172.21.15.94:0/775066205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b56a0 0x7f05800b9780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05800d40b0 tx=0x7f057402a150 comp rx=0 tx=0).stop 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 -- 172.21.15.94:0/775066205 shutdown_connections 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 --2- 172.21.15.94:0/775066205 >> v2:172.21.15.138:3300/0 conn(0x7f05800b9cc0 0x7f05800d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 --2- 172.21.15.94:0/775066205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b56a0 0x7f05800b9780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 -- 172.21.15.94:0/775066205 >> 172.21.15.94:0/775066205 conn(0x7f05800b7e60 msgr2=0x7f05800b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:57.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 -- 172.21.15.94:0/775066205 shutdown_connections 2024-04-25T16:26:57.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.671+0000 7f058554d640 1 -- 172.21.15.94:0/775066205 wait complete. 2024-04-25T16:26:57.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.673+0000 7f058554d640 1 Processor -- start 2024-04-25T16:26:57.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.673+0000 7f058554d640 1 -- start start 2024-04-25T16:26:57.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f058554d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05800b56a0 0x7f05801595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:57.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f058554d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 0x7f058015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:57.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f058554d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05800d4aa0 0x7f05800b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:57.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f058554d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05800ba0d0 con 0x7f05800b9cc0 2024-04-25T16:26:57.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f058554d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f058006c570 con 0x7f05800d4aa0 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f058554d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05800ba250 con 0x7f05800b56a0 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 0x7f058015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 0x7f058015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34942/0 (socket says 172.21.15.94:34942) 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 -- 172.21.15.94:0/2278381362 learned_addr learned my addr 172.21.15.94:0/2278381362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057effd640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05800b56a0 0x7f05801595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057f7fe640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05800d4aa0 0x7f05800b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 -- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05800b56a0 msgr2=0x7f05801595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05800b56a0 0x7f05801595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 -- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05800d4aa0 msgr2=0x7f05800b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05800d4aa0 0x7f05800b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 -- 172.21.15.94:0/2278381362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0580160c30 con 0x7f05800b9cc0 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057effd640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05800b56a0 0x7f05801595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057f7fe640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05800d4aa0 0x7f05800b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:57.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f057e7fc640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 0x7f058015b1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f057402a190 tx=0x7f05740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:57.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.674+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574041020 con 0x7f05800b9cc0 2024-04-25T16:26:57.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.675+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05740306c0 con 0x7f05800b9cc0 2024-04-25T16:26:57.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.675+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0580160f10 con 0x7f05800b9cc0 2024-04-25T16:26:57.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.675+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574037040 con 0x7f05800b9cc0 2024-04-25T16:26:57.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.675+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058015b720 con 0x7f05800b9cc0 2024-04-25T16:26:57.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.676+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058015cfc0 con 0x7f05800b9cc0 2024-04-25T16:26:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.676+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0574044020 con 0x7f05800b9cc0 2024-04-25T16:26:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.677+0000 7f05677fe640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f056c0744b0 0x7f056c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.677+0000 7f057effd640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f056c0744b0 0x7f056c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.677+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f05740b5b60 con 0x7f05800b9cc0 2024-04-25T16:26:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.677+0000 7f057effd640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f056c0744b0 0x7f056c0769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0570001d60 tx=0x7f0570009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:57.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.681+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f057404b030 con 0x7f05800b9cc0 2024-04-25T16:26:57.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:57.857+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f05800cb680 con 0x7f05800b9cc0 2024-04-25T16:26:59.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.562+0000 7f05677fe640 1 -- 172.21.15.94:0/2278381362 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f0574080250 con 0x7f05800b9cc0 2024-04-25T16:26:59.563 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data_ec' created 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f056c0744b0 msgr2=0x7f056c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f056c0744b0 0x7f056c0769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0570001d60 tx=0x7f0570009040 comp rx=0 tx=0).stop 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 msgr2=0x7f058015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 0x7f058015b1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f057402a190 tx=0x7f05740046c0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 shutdown_connections 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f056c0744b0 0x7f056c0769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05800d4aa0 0x7f05800b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800b9cc0 0x7f058015b1e0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f057402a190 tx=0x7f05740046c0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 --2- 172.21.15.94:0/2278381362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05800b56a0 0x7f05801595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 >> 172.21.15.94:0/2278381362 conn(0x7f05800b7e60 msgr2=0x7f05800b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 shutdown_connections 2024-04-25T16:26:59.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.566+0000 7f058554d640 1 -- 172.21.15.94:0/2278381362 wait complete. 2024-04-25T16:26:59.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T16:26:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.701+0000 7fc970da8640 1 Processor -- start 2024-04-25T16:26:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.701+0000 7fc970da8640 1 -- start start 2024-04-25T16:26:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.702+0000 7fc970da8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 0x7fc96c1649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.702+0000 7fc970da8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c0d4f80 con 0x7fc96c0d4a80 2024-04-25T16:26:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.702+0000 7fc970da8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c0d5260 con 0x7fc96c06c7e0 2024-04-25T16:26:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.702+0000 7fc970da8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c0d1e00 con 0x7fc96c1608b0 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96a59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc96c0d4a80 0x7fc96c1642b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42056/0 (socket says 172.21.15.94:42056) 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96a59c640 1 -- 172.21.15.94:0/371572073 learned_addr learned my addr 172.21.15.94:0/371572073 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504175714 0 0) 0x7fc96c0d4f80 con 0x7fc96c0d4a80 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc954003520 con 0x7fc96c0d4a80 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc969d9b640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 0x7fc96c1649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641032770 0 0) 0x7fc954003520 con 0x7fc96c0d4a80 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 msgr2=0x7fc96c1649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 0x7fc96c1649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 >> v1:172.21.15.138:6789/0 conn(0x7fc96c06c7e0 legacy=0x7fc96c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0d5540 con 0x7fc96c0d4a80 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fc95c0019f0 con 0x7fc96c0d4a80 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 >> v1:172.21.15.94:6789/0 conn(0x7fc96c0d4a80 legacy=0x7fc96c1642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc954003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 0x7fc954004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc954004d40 0x7fc954005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c0d1e00 con 0x7fc9540039e0 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc95400baf0 con 0x7fc954004d40 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc95400bcf0 con 0x7fc96c1608b0 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96a59c640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc954003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc96ad9d640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc954004d40 0x7fc954005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.703+0000 7fc969d9b640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 0x7fc954004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96a59c640 1 -- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 msgr2=0x7fc954004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96a59c640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c1608b0 0x7fc954004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96a59c640 1 -- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc954004d40 msgr2=0x7fc954005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96a59c640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc954004d40 0x7fc954005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96a59c640 1 -- 172.21.15.94:0/371572073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc954003520 con 0x7fc9540039e0 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96ad9d640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc954004d40 0x7fc954005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96a59c640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc954003e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0b9530 tx=0x7fc954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4019675d073dc03b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00a4a0 con 0x7fc9540039e0 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c02f070 con 0x7fc9540039e0 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.704+0000 7fc96959a640 1 -- 172.21.15.94:0/371572073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c02bbe0 con 0x7fc9540039e0 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc970da8640 1 -- 172.21.15.94:0/371572073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 msgr2=0x7fc954003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc970da8640 1 --2- 172.21.15.94:0/371572073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc954003e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0b9530 tx=0x7fc954005830 comp rx=0 tx=0).stop 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc96ad9d640 1 -- 172.21.15.94:0/371572073 reap_dead start 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc970da8640 1 -- 172.21.15.94:0/371572073 shutdown_connections 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc970da8640 1 -- 172.21.15.94:0/371572073 >> 172.21.15.94:0/371572073 conn(0x7fc96c0d6a00 msgr2=0x7fc96c0b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc970da8640 1 -- 172.21.15.94:0/371572073 shutdown_connections 2024-04-25T16:26:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.705+0000 7fc970da8640 1 -- 172.21.15.94:0/371572073 wait complete. 2024-04-25T16:26:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.707+0000 7fc970da8640 1 Processor -- start 2024-04-25T16:26:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.707+0000 7fc970da8640 1 -- start start 2024-04-25T16:26:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc970da8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc96c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc970da8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c0cdcc0 0x7fc96c1619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc970da8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c161ef0 0x7fc96c1f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc970da8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c0b5950 con 0x7fc9540039e0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc970da8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c0d5b00 con 0x7fc96c0cdcc0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc970da8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c0b5ad0 con 0x7fc96c161ef0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc96a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc96c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc96a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc96c0cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34952/0 (socket says 172.21.15.94:34952) 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.708+0000 7fc96a59c640 1 -- 172.21.15.94:0/1392866814 learned_addr learned my addr 172.21.15.94:0/1392866814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc969d9b640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c0cdcc0 0x7fc96c1619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96ad9d640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c161ef0 0x7fc96c1f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96a59c640 1 -- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c161ef0 msgr2=0x7fc96c1f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96a59c640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c161ef0 0x7fc96c1f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96a59c640 1 -- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c0cdcc0 msgr2=0x7fc96c1619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96a59c640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c0cdcc0 0x7fc96c1619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96a59c640 1 -- 172.21.15.94:0/1392866814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0d12a0 con 0x7fc9540039e0 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc969d9b640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c0cdcc0 0x7fc96c1619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96ad9d640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c161ef0 0x7fc96c1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:26:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc96a59c640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc96c0cbfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00a880 tx=0x7fc95c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c02cdc0 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c00a480 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c034670 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c0cb800 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.709+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c0cbac0 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.710+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c06a180 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.711+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc95c034810 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.711+0000 7fc95affd640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc94c0744b0 0x7fc94c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.711+0000 7fc969d9b640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc94c0744b0 0x7fc94c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.711+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fc95c0b6170 con 0x7fc9540039e0 2024-04-25T16:26:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.712+0000 7fc969d9b640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc94c0744b0 0x7fc94c0769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc960009160 tx=0x7fc960008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:26:59.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.716+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc95c0805e0 con 0x7fc9540039e0 2024-04-25T16:26:59.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:26:59.893+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc96c0d4630 con 0x7fc9540039e0 2024-04-25T16:27:00.062 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-04-25T16:27:00.062+0000 7fa2fcbda640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T16:27:00.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.564+0000 7fc95affd640 1 -- 172.21.15.94:0/1392866814 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7fc95c07fee0 con 0x7fc9540039e0 2024-04-25T16:27:00.565 INFO:teuthology.orchestra.run.smithi094.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T16:27:00.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc94c0744b0 msgr2=0x7fc94c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc94c0744b0 0x7fc94c0769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc960009160 tx=0x7fc960008040 comp rx=0 tx=0).stop 2024-04-25T16:27:00.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 msgr2=0x7fc96c0cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc96c0cbfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00a880 tx=0x7fc95c001dc0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 shutdown_connections 2024-04-25T16:27:00.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc94c0744b0 0x7fc94c0769c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96c161ef0 0x7fc96c1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c0cdcc0 0x7fc96c1619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 --2- 172.21.15.94:0/1392866814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9540039e0 0x7fc96c0cbfe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.567+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 >> 172.21.15.94:0/1392866814 conn(0x7fc96c0d6a00 msgr2=0x7fc96c0b7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.570+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 shutdown_connections 2024-04-25T16:27:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.570+0000 7fc970da8640 1 -- 172.21.15.94:0/1392866814 wait complete. 2024-04-25T16:27:00.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T16:27:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.713+0000 7f996e442640 1 Processor -- start 2024-04-25T16:27:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.713+0000 7f996e442640 1 -- start start 2024-04-25T16:27:00.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f996e442640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f996816c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:00.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f996e442640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99680d39e0 con 0x7f9968164e40 2024-04-25T16:27:00.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f996e442640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99680d2dd0 con 0x7f99680b9780 2024-04-25T16:27:00.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f996e442640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f99680d30b0 con 0x7f99680cf970 2024-04-25T16:27:00.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f99677fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9968164e40 0x7f9968165250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42070/0 (socket says 172.21.15.94:42070) 2024-04-25T16:27:00.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f99677fe640 1 -- 172.21.15.94:0/3782852026 learned_addr learned my addr 172.21.15.94:0/3782852026 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f996c9df640 1 --2- 172.21.15.94:0/3782852026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f996816c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2853069734 0 0) 0x7f99680d39e0 con 0x7f9968164e40 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.714+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9950003520 con 0x7f9968164e40 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766208314 0 0) 0x7f99680d2dd0 con 0x7f99680b9780 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99680d39e0 con 0x7f99680b9780 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 423608123 0 0) 0x7f9950003520 con 0x7f9968164e40 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f996c9df640 1 -- 172.21.15.94:0/3782852026 >> v1:172.21.15.138:6789/0 conn(0x7f99680b9780 legacy=0x7f996809c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f996c9df640 1 -- 172.21.15.94:0/3782852026 >> v1:172.21.15.94:6789/0 conn(0x7f9968164e40 legacy=0x7f9968165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f996c9df640 1 -- 172.21.15.94:0/3782852026 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99680d3390 con 0x7f99680cf970 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.715+0000 7f996c9df640 1 --2- 172.21.15.94:0/3782852026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f996816c100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9960009dc0 tx=0x7f996002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b378bcd42989d690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c001690 con 0x7f99680cf970 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960007540 con 0x7f99680cf970 2024-04-25T16:27:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f9966ffd640 1 -- 172.21.15.94:0/3782852026 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600078f0 con 0x7f99680cf970 2024-04-25T16:27:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 -- 172.21.15.94:0/3782852026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 msgr2=0x7f996816c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 --2- 172.21.15.94:0/3782852026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f996816c100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9960009dc0 tx=0x7f996002a660 comp rx=0 tx=0).stop 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 -- 172.21.15.94:0/3782852026 shutdown_connections 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 --2- 172.21.15.94:0/3782852026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f996816c100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 -- 172.21.15.94:0/3782852026 >> 172.21.15.94:0/3782852026 conn(0x7f99680b5b20 msgr2=0x7f99680b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 -- 172.21.15.94:0/3782852026 shutdown_connections 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.716+0000 7f996e442640 1 -- 172.21.15.94:0/3782852026 wait complete. 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.718+0000 7f996e442640 1 Processor -- start 2024-04-25T16:27:00.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.718+0000 7f996e442640 1 -- start start 2024-04-25T16:27:00.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.719+0000 7f996e442640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99680b9780 0x7f996815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:00.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.719+0000 7f996e442640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f99681f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:00.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.719+0000 7f996e442640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 0x7f99680b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.719+0000 7f996e442640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99680b4f60 con 0x7f9968164e40 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.719+0000 7f996e442640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f996809c9c0 con 0x7f99680b9780 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.719+0000 7f996e442640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f99680b50e0 con 0x7f99680cf970 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 0x7f99680b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 0x7f99680b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34956/0 (socket says 172.21.15.94:34956) 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 -- 172.21.15.94:0/3982723918 learned_addr learned my addr 172.21.15.94:0/3982723918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f9967fff640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99680b9780 0x7f996815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f99677fe640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f99681f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 -- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 msgr2=0x7f99681f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f99681f66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 -- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99680b9780 msgr2=0x7f996815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99680b9780 0x7f996815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:00.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 -- 172.21.15.94:0/3982723918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996815ec00 con 0x7f9968164e40 2024-04-25T16:27:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996c9df640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 0x7f99680b1cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9960009ee0 tx=0x7f9960004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960040070 con 0x7f9968164e40 2024-04-25T16:27:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960031bc0 con 0x7f9968164e40 2024-04-25T16:27:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996815eee0 con 0x7f9968164e40 2024-04-25T16:27:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960009450 con 0x7f9968164e40 2024-04-25T16:27:00.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.720+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996815f4a0 con 0x7f9968164e40 2024-04-25T16:27:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.721+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996815e970 con 0x7f9968164e40 2024-04-25T16:27:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.722+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9960005d10 con 0x7f9968164e40 2024-04-25T16:27:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.723+0000 7f9964ff9640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f99540744b0 0x7f99540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.723+0000 7f9967fff640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f99540744b0 0x7f99540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.723+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f99600b64a0 con 0x7f9968164e40 2024-04-25T16:27:00.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.723+0000 7f9967fff640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f99540744b0 0x7f99540769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99580011e0 tx=0x7f9958009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:00.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.727+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f996004c020 con 0x7f9968164e40 2024-04-25T16:27:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:00.939+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f996806a180 con 0x7f9968164e40 2024-04-25T16:27:01.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.590+0000 7f9964ff9640 1 -- 172.21.15.94:0/3982723918 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f9960080240 con 0x7f9968164e40 2024-04-25T16:27:01.591 INFO:teuthology.orchestra.run.smithi094.stderr:added data pool 4 to fsmap 2024-04-25T16:27:01.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f99540744b0 msgr2=0x7f99540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f99540744b0 0x7f99540769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99580011e0 tx=0x7f9958009040 comp rx=0 tx=0).stop 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 msgr2=0x7f99680b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 0x7f99680b1cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9960009ee0 tx=0x7f9960004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 shutdown_connections 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f99540744b0 0x7f99540769c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.592+0000 7f996e442640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9968164e40 0x7f99680b1cc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.593+0000 7f996e442640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99680cf970 0x7f99681f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.593+0000 7f996e442640 1 --2- 172.21.15.94:0/3982723918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99680b9780 0x7f996815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.593+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 >> 172.21.15.94:0/3982723918 conn(0x7f99680b5b20 msgr2=0x7f99680727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.593+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 shutdown_connections 2024-04-25T16:27:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.593+0000 7f996e442640 1 -- 172.21.15.94:0/3982723918 wait complete. 2024-04-25T16:27:01.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:27:01.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.713+0000 7fec8633f640 1 Processor -- start 2024-04-25T16:27:01.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.713+0000 7fec8633f640 1 -- start start 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.714+0000 7fec8633f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fec800b97e0 0x7fec8006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.714+0000 7fec8633f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.714+0000 7fec8633f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec800d3b80 con 0x7fec8006e630 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.714+0000 7fec8633f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fec8009d6d0 con 0x7fec800b97e0 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.714+0000 7fec8633f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec8009d3f0 con 0x7fec80165350 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7ffff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fec800b97e0 0x7fec8006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7f7fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fec8006e630 0x7fec80164c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:36900/0 (socket says 172.21.15.94:36900) 2024-04-25T16:27:01.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec848dc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:01.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7f7fe640 1 -- 172.21.15.94:0/3826292200 learned_addr learned my addr 172.21.15.94:0/3826292200 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7ffff640 1 -- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 msgr2=0x7fec800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7ffff640 1 --2- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7ffff640 1 -- 172.21.15.94:0/3826292200 >> v1:172.21.15.138:6789/0 conn(0x7fec8006e630 legacy=0x7fec80164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec7ffff640 1 -- 172.21.15.94:0/3826292200 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec800d3530 con 0x7fec800b97e0 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.715+0000 7fec848dc640 1 --2- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.716+0000 7fec7ffff640 1 --2- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec800b97e0 0x7fec8006e0f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fec700097a0 tx=0x7fec7002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5496c82d9ca0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.716+0000 7fec7effd640 1 -- 172.21.15.94:0/3826292200 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70037070 con 0x7fec800b97e0 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.716+0000 7fec7effd640 1 -- 172.21.15.94:0/3826292200 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec70032440 con 0x7fec800b97e0 2024-04-25T16:27:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.716+0000 7fec7effd640 1 -- 172.21.15.94:0/3826292200 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70032810 con 0x7fec800b97e0 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 -- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec800b97e0 msgr2=0x7fec8006e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 --2- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec800b97e0 0x7fec8006e0f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fec700097a0 tx=0x7fec7002a180 comp rx=0 tx=0).stop 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 -- 172.21.15.94:0/3826292200 shutdown_connections 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 --2- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 --2- 172.21.15.94:0/3826292200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec800b97e0 0x7fec8006e0f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 -- 172.21.15.94:0/3826292200 >> 172.21.15.94:0/3826292200 conn(0x7fec800b5a70 msgr2=0x7fec800b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 -- 172.21.15.94:0/3826292200 shutdown_connections 2024-04-25T16:27:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.717+0000 7fec8633f640 1 -- 172.21.15.94:0/3826292200 wait complete. 2024-04-25T16:27:01.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.718+0000 7fec8633f640 1 Processor -- start 2024-04-25T16:27:01.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.719+0000 7fec8633f640 1 -- start start 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec8633f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec8006e630 0x7fec80157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec8633f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 0x7fec800cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec8633f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec8633f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec800b2bf0 con 0x7fec800b97e0 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec8633f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec8006f4e0 con 0x7fec8006e630 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec8633f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec800b2d70 con 0x7fec80165350 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 0x7fec800cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 0x7fec800cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47148/0 (socket says 172.21.15.94:47148) 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 -- 172.21.15.94:0/1226044001 learned_addr learned my addr 172.21.15.94:0/1226044001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec848dc640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7ffff640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec8006e630 0x7fec80157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 -- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 msgr2=0x7fec800b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 -- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec8006e630 msgr2=0x7fec80157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec8006e630 0x7fec80157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 -- 172.21.15.94:0/1226044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec800cb8f0 con 0x7fec800b97e0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.720+0000 7fec7f7fe640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 0x7fec800cafc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c3c0 tx=0x7fec7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.721+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74017070 con 0x7fec800b97e0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.721+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74005ce0 con 0x7fec800b97e0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.721+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec800cbbd0 con 0x7fec800b97e0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.721+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec740072f0 con 0x7fec800b97e0 2024-04-25T16:27:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.721+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec800cc190 con 0x7fec800b97e0 2024-04-25T16:27:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.722+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8006a180 con 0x7fec800b97e0 2024-04-25T16:27:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.722+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec7401a070 con 0x7fec800b97e0 2024-04-25T16:27:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.723+0000 7fec7cff9640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fec6c0744b0 0x7fec6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.723+0000 7fec7ffff640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fec6c0744b0 0x7fec6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.723+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fec74094860 con 0x7fec800b97e0 2024-04-25T16:27:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.723+0000 7fec7ffff640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fec6c0744b0 0x7fec6c0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec70034000 tx=0x7fec7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:01.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.727+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec7405e670 con 0x7fec800b97e0 2024-04-25T16:27:01.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.937+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec800c83e0 con 0x7fec800b97e0 2024-04-25T16:27:01.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.937+0000 7fec7cff9640 1 -- 172.21.15.94:0/1226044001 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7fec74062970 con 0x7fec800b97e0 2024-04-25T16:27:01.938 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:01.938 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:00.941324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:27:01.939 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-04-25T16:27:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fec6c0744b0 msgr2=0x7fec6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fec6c0744b0 0x7fec6c0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec70034000 tx=0x7fec7002e000 comp rx=0 tx=0).stop 2024-04-25T16:27:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 msgr2=0x7fec800cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 0x7fec800cafc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c3c0 tx=0x7fec7400c880 comp rx=0 tx=0).stop 2024-04-25T16:27:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 shutdown_connections 2024-04-25T16:27:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fec6c0744b0 0x7fec6c0769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec80165350 0x7fec800b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800b97e0 0x7fec800cafc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 --2- 172.21.15.94:0/1226044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec8006e630 0x7fec80157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 >> 172.21.15.94:0/1226044001 conn(0x7fec800b5a70 msgr2=0x7fec8006ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 shutdown_connections 2024-04-25T16:27:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:01.939+0000 7fec8633f640 1 -- 172.21.15.94:0/1226044001 wait complete. 2024-04-25T16:27:01.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:02.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.057+0000 7fa806e20640 1 Processor -- start 2024-04-25T16:27:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.057+0000 7fa806e20640 1 -- start start 2024-04-25T16:27:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa806e20640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa8000d4dc0 0x7fa80006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa806e20640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8001607f0 0x7fa8000d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa806e20640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80006dec0 con 0x7fa800160c00 2024-04-25T16:27:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa806e20640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa80006c8e0 con 0x7fa8000d4dc0 2024-04-25T16:27:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa806e20640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa80006cbc0 con 0x7fa8001607f0 2024-04-25T16:27:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa804bbc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa8000d4dc0 0x7fa80006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa804bbc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa8000d4dc0 0x7fa80006d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47156/0 (socket says 172.21.15.94:47156) 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa804bbc640 1 -- 172.21.15.94:0/4287049247 learned_addr learned my addr 172.21.15.94:0/4287049247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.058+0000 7fa7fffff640 1 --2- 172.21.15.94:0/4287049247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8001607f0 0x7fa8000d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa804bbc640 1 -- 172.21.15.94:0/4287049247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8001607f0 msgr2=0x7fa8000d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa804bbc640 1 --2- 172.21.15.94:0/4287049247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8001607f0 0x7fa8000d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa804bbc640 1 -- 172.21.15.94:0/4287049247 >> v1:172.21.15.138:6789/0 conn(0x7fa800160c00 legacy=0x7fa8000b9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa804bbc640 1 -- 172.21.15.94:0/4287049247 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80006cea0 con 0x7fa8000d4dc0 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa804bbc640 1 --2- 172.21.15.94:0/4287049247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8000d4dc0 0x7fa80006d980 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009870 tx=0x7fa7f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5eaca139b6f92d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa7ff7fe640 1 -- 172.21.15.94:0/4287049247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4037070 con 0x7fa8000d4dc0 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa7ff7fe640 1 -- 172.21.15.94:0/4287049247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4032440 con 0x7fa8000d4dc0 2024-04-25T16:27:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.059+0000 7fa7ff7fe640 1 -- 172.21.15.94:0/4287049247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4032810 con 0x7fa8000d4dc0 2024-04-25T16:27:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 -- 172.21.15.94:0/4287049247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8000d4dc0 msgr2=0x7fa80006d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 --2- 172.21.15.94:0/4287049247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8000d4dc0 0x7fa80006d980 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009870 tx=0x7fa7f402a1c0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 -- 172.21.15.94:0/4287049247 shutdown_connections 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 --2- 172.21.15.94:0/4287049247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8001607f0 0x7fa8000d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 --2- 172.21.15.94:0/4287049247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8000d4dc0 0x7fa80006d980 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 -- 172.21.15.94:0/4287049247 >> 172.21.15.94:0/4287049247 conn(0x7fa8000d3380 msgr2=0x7fa8000d62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 -- 172.21.15.94:0/4287049247 shutdown_connections 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.060+0000 7fa806e20640 1 -- 172.21.15.94:0/4287049247 wait complete. 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.062+0000 7fa806e20640 1 Processor -- start 2024-04-25T16:27:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.062+0000 7fa806e20640 1 -- start start 2024-04-25T16:27:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa806e20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8000d4dc0 0x7fa800155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa806e20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 0x7fa8000cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa806e20640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa800160c00 0x7fa8000cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa806e20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa80006c570 con 0x7fa8001607f0 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa806e20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa80006d460 con 0x7fa8000d4dc0 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa806e20640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000d04f0 con 0x7fa800160c00 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 0x7fa8000cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 0x7fa8000cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47164/0 (socket says 172.21.15.94:47164) 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.063+0000 7fa7fffff640 1 -- 172.21.15.94:0/598275238 learned_addr learned my addr 172.21.15.94:0/598275238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa804bbc640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8000d4dc0 0x7fa800155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa8053bd640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa800160c00 0x7fa8000cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fffff640 1 -- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa800160c00 msgr2=0x7fa8000cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fffff640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa800160c00 0x7fa8000cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fffff640 1 -- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8000d4dc0 msgr2=0x7fa800155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fffff640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8000d4dc0 0x7fa800155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fffff640 1 -- 172.21.15.94:0/598275238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa800155680 con 0x7fa8001607f0 2024-04-25T16:27:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fffff640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 0x7fa8000cd9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d5f0 tx=0x7fa7ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0125f0 con 0x7fa8001607f0 2024-04-25T16:27:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ec019070 con 0x7fa8001607f0 2024-04-25T16:27:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80014b7a0 con 0x7fa8001607f0 2024-04-25T16:27:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0129d0 con 0x7fa8001607f0 2024-04-25T16:27:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.064+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80014bd60 con 0x7fa8001607f0 2024-04-25T16:27:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.065+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80014c420 con 0x7fa8001607f0 2024-04-25T16:27:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.066+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa7ec012b70 con 0x7fa8001607f0 2024-04-25T16:27:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.066+0000 7fa7fd7fa640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7e40744e0 0x7fa7e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.066+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa7ec0958a0 con 0x7fa8001607f0 2024-04-25T16:27:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.066+0000 7fa804bbc640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7e40744e0 0x7fa7e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.067+0000 7fa804bbc640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7e40744e0 0x7fa7e40769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4034000 tx=0x7fa7f402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.071+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa80014c420 con 0x7fa8001607f0 2024-04-25T16:27:02.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.245+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa80006a180 con 0x7fa8001607f0 2024-04-25T16:27:02.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.246+0000 7fa7fd7fa640 1 -- 172.21.15.94:0/598275238 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fa7ec05fd80 con 0x7fa8001607f0 2024-04-25T16:27:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:02.247 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7e40744e0 msgr2=0x7fa7e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7e40744e0 0x7fa7e40769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4034000 tx=0x7fa7f402e000 comp rx=0 tx=0).stop 2024-04-25T16:27:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 msgr2=0x7fa8000cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 0x7fa8000cd9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d5f0 tx=0x7fa7ec00dab0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 shutdown_connections 2024-04-25T16:27:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7e40744e0 0x7fa7e40769f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa800160c00 0x7fa8000cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8001607f0 0x7fa8000cd9d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 --2- 172.21.15.94:0/598275238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8000d4dc0 0x7fa800155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 >> 172.21.15.94:0/598275238 conn(0x7fa8000d3380 msgr2=0x7fa8000cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 shutdown_connections 2024-04-25T16:27:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.248+0000 7fa806e20640 1 -- 172.21.15.94:0/598275238 wait complete. 2024-04-25T16:27:02.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.367+0000 7f1e878b1640 1 Processor -- start 2024-04-25T16:27:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.367+0000 7f1e878b1640 1 -- start start 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e878b1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e800d11b0 0x7f1e800b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e878b1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e800d24a0 con 0x7f1e8008e080 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e878b1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e800d2720 con 0x7f1e800d2090 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e878b1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1e800d2a00 con 0x7f1e800d11b0 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e85e4e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1e8008e080 0x7f1e800cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53036/0 (socket says 172.21.15.94:53036) 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e85e4e640 1 -- 172.21.15.94:0/193052992 learned_addr learned my addr 172.21.15.94:0/193052992 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e84e4c640 1 --2- 172.21.15.94:0/193052992 >> v2:172.21.15.138:3300/0 conn(0x7f1e800d11b0 0x7f1e800b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630203135 0 0) 0x7f1e78001110 con 0x7f1e8008e080 2024-04-25T16:27:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e6c003520 con 0x7f1e8008e080 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367257868 0 0) 0x7f1e800d2720 con 0x7f1e800d2090 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e78001110 con 0x7f1e800d2090 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2727243436 0 0) 0x7f1e6c003520 con 0x7f1e8008e080 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e84e4c640 1 -- 172.21.15.94:0/193052992 >> v1:172.21.15.138:6789/0 conn(0x7f1e800d2090 legacy=0x7f1e800d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e84e4c640 1 -- 172.21.15.94:0/193052992 >> v1:172.21.15.94:6789/0 conn(0x7f1e8008e080 legacy=0x7f1e800cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.369+0000 7f1e84e4c640 1 -- 172.21.15.94:0/193052992 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e800d2ce0 con 0x7f1e800d11b0 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.370+0000 7f1e84e4c640 1 --2- 172.21.15.94:0/193052992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e800d11b0 0x7f1e800b3350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e700097a0 tx=0x7f1e7002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3637a848a734bb04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.370+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70037070 con 0x7f1e800d11b0 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.370+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7002abc0 con 0x7f1e800d11b0 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.370+0000 7f1e77fff640 1 -- 172.21.15.94:0/193052992 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78001840 con 0x7f1e800d11b0 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 -- 172.21.15.94:0/193052992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e800d11b0 msgr2=0x7f1e800b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 --2- 172.21.15.94:0/193052992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e800d11b0 0x7f1e800b3350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e700097a0 tx=0x7f1e7002a130 comp rx=0 tx=0).stop 2024-04-25T16:27:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 -- 172.21.15.94:0/193052992 shutdown_connections 2024-04-25T16:27:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 --2- 172.21.15.94:0/193052992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e800d11b0 0x7f1e800b3350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 -- 172.21.15.94:0/193052992 >> 172.21.15.94:0/193052992 conn(0x7f1e800d4090 msgr2=0x7f1e800d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 -- 172.21.15.94:0/193052992 shutdown_connections 2024-04-25T16:27:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.371+0000 7f1e878b1640 1 -- 172.21.15.94:0/193052992 wait complete. 2024-04-25T16:27:02.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.373+0000 7f1e878b1640 1 Processor -- start 2024-04-25T16:27:02.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.373+0000 7f1e878b1640 1 -- start start 2024-04-25T16:27:02.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e878b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e8008e080 0x7f1e800a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e878b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 0x7f1e800cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e878b1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e800cd3b0 0x7f1e8015fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e878b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e800ced50 con 0x7f1e800d2090 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e878b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8008e490 con 0x7f1e8008e080 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e878b1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e800b4470 con 0x7f1e800cd3b0 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e84e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 0x7f1e800cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e84e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 0x7f1e800cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47170/0 (socket says 172.21.15.94:47170) 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e84e4c640 1 -- 172.21.15.94:0/3050344345 learned_addr learned my addr 172.21.15.94:0/3050344345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e85e4e640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e800cd3b0 0x7f1e8015fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.374+0000 7f1e8564d640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e8008e080 0x7f1e800a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e84e4c640 1 -- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e800cd3b0 msgr2=0x7f1e8015fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e84e4c640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e800cd3b0 0x7f1e8015fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e84e4c640 1 -- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e8008e080 msgr2=0x7f1e800a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e84e4c640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e8008e080 0x7f1e800a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e84e4c640 1 -- 172.21.15.94:0/3050344345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e800cd7c0 con 0x7f1e800d2090 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e8564d640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e8008e080 0x7f1e800a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e85e4e640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e800cd3b0 0x7f1e8015fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e84e4c640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 0x7f1e800cce70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e700098c0 tx=0x7f1e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70042020 con 0x7f1e800d2090 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7003d670 con 0x7f1e800d2090 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70037040 con 0x7f1e800d2090 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e80161bc0 con 0x7f1e800d2090 2024-04-25T16:27:02.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.375+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8015f300 con 0x7f1e800d2090 2024-04-25T16:27:02.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.376+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e80161930 con 0x7f1e800d2090 2024-04-25T16:27:02.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.377+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e70045020 con 0x7f1e800d2090 2024-04-25T16:27:02.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.377+0000 7f1e75ffb640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1e640744b0 0x7f1e640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.378+0000 7f1e8564d640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1e640744b0 0x7f1e640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.378+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1e700b5e30 con 0x7f1e800d2090 2024-04-25T16:27:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.378+0000 7f1e8564d640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1e640744b0 0x7f1e640769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c007fd0 tx=0x7f1e7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.382+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e7004c030 con 0x7f1e800d2090 2024-04-25T16:27:02.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.556+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e8006a180 con 0x7f1e800d2090 2024-04-25T16:27:02.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.557+0000 7f1e75ffb640 1 -- 172.21.15.94:0/3050344345 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f1e800cd7c0 con 0x7f1e800d2090 2024-04-25T16:27:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:02.558 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:02.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1e640744b0 msgr2=0x7f1e640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1e640744b0 0x7f1e640769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c007fd0 tx=0x7f1e7c009040 comp rx=0 tx=0).stop 2024-04-25T16:27:02.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 msgr2=0x7f1e800cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 0x7f1e800cce70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e700098c0 tx=0x7f1e700046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 shutdown_connections 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1e640744b0 0x7f1e640769c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e800cd3b0 0x7f1e8015fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e800d2090 0x7f1e800cce70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 --2- 172.21.15.94:0/3050344345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e8008e080 0x7f1e800a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 >> 172.21.15.94:0/3050344345 conn(0x7f1e800d4090 msgr2=0x7f1e800d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 shutdown_connections 2024-04-25T16:27:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.559+0000 7f1e878b1640 1 -- 172.21.15.94:0/3050344345 wait complete. 2024-04-25T16:27:02.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:27:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.691+0000 7f979f830640 1 Processor -- start 2024-04-25T16:27:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.691+0000 7f979f830640 1 -- start start 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.692+0000 7f979f830640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9798160790 0x7f9798164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.692+0000 7f979f830640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97980b5a30 con 0x7f97980b5230 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.692+0000 7f979f830640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97980d5610 con 0x7f979806cea0 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979f830640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f979806ca10 con 0x7f9798160790 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979d5cc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f97980b5230 0x7f9798164210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53046/0 (socket says 172.21.15.94:53046) 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979d5cc640 1 -- 172.21.15.94:0/3549798963 learned_addr learned my addr 172.21.15.94:0/3549798963 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945863804 0 0) 0x7f9794001490 con 0x7f97980b5230 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979cdcb640 1 --2- 172.21.15.94:0/3549798963 >> v2:172.21.15.138:3300/0 conn(0x7f9798160790 0x7f9798164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9784003520 con 0x7f97980b5230 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 188789100 0 0) 0x7f9784003520 con 0x7f97980b5230 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 >> v2:172.21.15.138:3300/0 conn(0x7f9798160790 msgr2=0x7f9798164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 --2- 172.21.15.94:0/3549798963 >> v2:172.21.15.138:3300/0 conn(0x7f9798160790 0x7f9798164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 >> v1:172.21.15.138:6789/0 conn(0x7f979806cea0 legacy=0x7f979806d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97980d58f0 con 0x7f97980b5230 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f9794001bc0 con 0x7f97980b5230 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 >> v1:172.21.15.94:6789/0 conn(0x7f97980b5230 legacy=0x7f9798164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f9784003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9784004800 0x7f9784004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97840052a0 0x7f978400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f979806ca10 con 0x7f9784003a00 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f978400c230 con 0x7f9784004800 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f978400c430 con 0x7f97840052a0 2024-04-25T16:27:02.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979d5cc640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f9784003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979cdcb640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9784004800 0x7f9784004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.693+0000 7f979ddcd640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97840052a0 0x7f978400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979d5cc640 1 -- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97840052a0 msgr2=0x7f978400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979d5cc640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97840052a0 0x7f978400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979d5cc640 1 -- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9784004800 msgr2=0x7f9784004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979d5cc640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9784004800 0x7f9784004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979ddcd640 1 -- 172.21.15.94:0/3549798963 reap_dead start 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979d5cc640 1 -- 172.21.15.94:0/3549798963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9784003520 con 0x7f9784003a00 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f979d5cc640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f9784003e70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f979400a930 tx=0x7f9784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719bff8eaadf8778 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400a580 con 0x7f9784003a00 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979402f070 con 0x7f9784003a00 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.694+0000 7f978ffff640 1 -- 172.21.15.94:0/3549798963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979402bdb0 con 0x7f9784003a00 2024-04-25T16:27:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 -- 172.21.15.94:0/3549798963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 msgr2=0x7f9784003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f9784003e70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f979400a930 tx=0x7f9784005830 comp rx=0 tx=0).stop 2024-04-25T16:27:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 -- 172.21.15.94:0/3549798963 shutdown_connections 2024-04-25T16:27:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 --2- 172.21.15.94:0/3549798963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f9784003e70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 -- 172.21.15.94:0/3549798963 >> 172.21.15.94:0/3549798963 conn(0x7f97980d3570 msgr2=0x7f97980b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 -- 172.21.15.94:0/3549798963 shutdown_connections 2024-04-25T16:27:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.695+0000 7f979f830640 1 -- 172.21.15.94:0/3549798963 wait complete. 2024-04-25T16:27:02.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.697+0000 7f979f830640 1 Processor -- start 2024-04-25T16:27:02.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.697+0000 7f979f830640 1 -- start start 2024-04-25T16:27:02.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979f830640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f97980ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979f830640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97980d13b0 0x7f97980d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979f830640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97981f9090 0x7f97981f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979f830640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97980b9870 con 0x7f9784003a00 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979f830640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97980b96f0 con 0x7f97981f9090 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979f830640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f979806d2b0 con 0x7f97980d13b0 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979d5cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f97980ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979d5cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f97980ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47192/0 (socket says 172.21.15.94:47192) 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.698+0000 7f979d5cc640 1 -- 172.21.15.94:0/1881136801 learned_addr learned my addr 172.21.15.94:0/1881136801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979ddcd640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97981f9090 0x7f97981f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979cdcb640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97980d13b0 0x7f97980d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979d5cc640 1 -- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97980d13b0 msgr2=0x7f97980d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979d5cc640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97980d13b0 0x7f97980d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979d5cc640 1 -- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97981f9090 msgr2=0x7f97981f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979d5cc640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97981f9090 0x7f97981f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979d5cc640 1 -- 172.21.15.94:0/1881136801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97981f99e0 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979ddcd640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97981f9090 0x7f97981f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979d5cc640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f97980ce950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f979400a930 tx=0x7f97980d5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400a580 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9794001c30 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97980cf380 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979402bd50 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.699+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97980cf940 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.700+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979806a180 con 0x7f9784003a00 2024-04-25T16:27:02.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.701+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9794046060 con 0x7f9784003a00 2024-04-25T16:27:02.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.701+0000 7f978dffb640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f977c0744b0 0x7f977c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:02.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.701+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f97940b5640 con 0x7f9784003a00 2024-04-25T16:27:02.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.701+0000 7f979cdcb640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f977c0744b0 0x7f977c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:02.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.702+0000 7f979cdcb640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f977c0744b0 0x7f977c0769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97880090e0 tx=0x7f9788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:02.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.706+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97980b5a30 con 0x7f9784003a00 2024-04-25T16:27:02.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.919+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f979815efc0 con 0x7f9784003a00 2024-04-25T16:27:02.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.919+0000 7f978dffb640 1 -- 172.21.15.94:0/1881136801 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7f979407fa90 con 0x7f9784003a00 2024-04-25T16:27:02.920 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:02.920 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:00.941324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:27:02.920 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-04-25T16:27:02.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f977c0744b0 msgr2=0x7f977c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f977c0744b0 0x7f977c0769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97880090e0 tx=0x7f9788008040 comp rx=0 tx=0).stop 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 msgr2=0x7f97980ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f97980ce950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f979400a930 tx=0x7f97980d5ac0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 shutdown_connections 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f977c0744b0 0x7f977c0769c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97981f9090 0x7f97981f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97980d13b0 0x7f97980d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 --2- 172.21.15.94:0/1881136801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9784003a00 0x7f97980ce950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 >> 172.21.15.94:0/1881136801 conn(0x7f97980d3570 msgr2=0x7f97980b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 shutdown_connections 2024-04-25T16:27:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:02.921+0000 7f979f830640 1 -- 172.21.15.94:0/1881136801 wait complete. 2024-04-25T16:27:02.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.040+0000 7fa1c9bb7640 1 Processor -- start 2024-04-25T16:27:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.040+0000 7fa1c9bb7640 1 -- start start 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c9bb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c406f4c0 0x7fa1c406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c9bb7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa1c40d2d70 0x7fa1c40d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c9bb7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa1c409b9b0 0x7fa1c409bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c9bb7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c406f8d0 con 0x7fa1c409b9b0 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c9bb7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c4162ac0 con 0x7fa1c40d2d70 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c9bb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c40d36c0 con 0x7fa1c406f4c0 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa1c409b9b0 0x7fa1c409bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa1c409b9b0 0x7fa1c409bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47196/0 (socket says 172.21.15.94:47196) 2024-04-25T16:27:03.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c3fff640 1 -- 172.21.15.94:0/31690615 learned_addr learned my addr 172.21.15.94:0/31690615 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c2ffd640 1 --2- 172.21.15.94:0/31690615 >> v2:172.21.15.138:3300/0 conn(0x7fa1c40d2d70 0x7fa1c40d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.041+0000 7fa1c37fe640 1 --2- 172.21.15.94:0/31690615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c406f4c0 0x7fa1c406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c3fff640 1 -- 172.21.15.94:0/31690615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c406f4c0 msgr2=0x7fa1c406ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c3fff640 1 --2- 172.21.15.94:0/31690615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c406f4c0 0x7fa1c406ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c3fff640 1 -- 172.21.15.94:0/31690615 >> v2:172.21.15.138:3300/0 conn(0x7fa1c40d2d70 msgr2=0x7fa1c40d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c3fff640 1 --2- 172.21.15.94:0/31690615 >> v2:172.21.15.138:3300/0 conn(0x7fa1c40d2d70 0x7fa1c40d3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c3fff640 1 -- 172.21.15.94:0/31690615 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c406d500 con 0x7fa1c409b9b0 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c2ffd640 1 --2- 172.21.15.94:0/31690615 >> v2:172.21.15.138:3300/0 conn(0x7fa1c40d2d70 0x7fa1c40d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c3fff640 1 --2- 172.21.15.94:0/31690615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c409b9b0 0x7fa1c409bdc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4063be0 tx=0x7fa1bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b702937938389689 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c27fc640 1 -- 172.21.15.94:0/31690615 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc036070 con 0x7fa1c409b9b0 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c27fc640 1 -- 172.21.15.94:0/31690615 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1bc031600 con 0x7fa1c409b9b0 2024-04-25T16:27:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.042+0000 7fa1c27fc640 1 -- 172.21.15.94:0/31690615 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0319d0 con 0x7fa1c409b9b0 2024-04-25T16:27:03.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/31690615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c409b9b0 msgr2=0x7fa1c409bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/31690615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c409b9b0 0x7fa1c409bdc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4063be0 tx=0x7fa1bc02a6a0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/31690615 shutdown_connections 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/31690615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c409b9b0 0x7fa1c409bdc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/31690615 >> v2:172.21.15.138:3300/0 conn(0x7fa1c40d2d70 0x7fa1c40d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/31690615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c406f4c0 0x7fa1c406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/31690615 >> 172.21.15.94:0/31690615 conn(0x7fa1c40b7cc0 msgr2=0x7fa1c40b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/31690615 shutdown_connections 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.043+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/31690615 wait complete. 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.045+0000 7fa1c9bb7640 1 Processor -- start 2024-04-25T16:27:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.045+0000 7fa1c9bb7640 1 -- start start 2024-04-25T16:27:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c9bb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 0x7fa1c41f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c9bb7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c409b9b0 0x7fa1c41fa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c9bb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c40d2d70 0x7fa1c41fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c9bb7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4162750 con 0x7fa1c406f4c0 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c9bb7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4162470 con 0x7fa1c409b9b0 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c9bb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c4161c70 con 0x7fa1c40d2d70 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.046+0000 7fa1c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 0x7fa1c41f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 0x7fa1c41f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47198/0 (socket says 172.21.15.94:47198) 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 -- 172.21.15.94:0/3344031163 learned_addr learned my addr 172.21.15.94:0/3344031163 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c2ffd640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c409b9b0 0x7fa1c41fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c3fff640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c40d2d70 0x7fa1c41fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 -- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c40d2d70 msgr2=0x7fa1c41fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c40d2d70 0x7fa1c41fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 -- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c409b9b0 msgr2=0x7fa1c41fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c409b9b0 0x7fa1c41fa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 -- 172.21.15.94:0/3344031163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c41608a0 con 0x7fa1c406f4c0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c37fe640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 0x7fa1c41f9af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d620 tx=0x7fa1b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4004240 con 0x7fa1c406f4c0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b40043e0 con 0x7fa1c406f4c0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4003610 con 0x7fa1c406f4c0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c415ef90 con 0x7fa1c406f4c0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.047+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c415f550 con 0x7fa1c406f4c0 2024-04-25T16:27:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.048+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c406a180 con 0x7fa1c406f4c0 2024-04-25T16:27:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.049+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1b401b020 con 0x7fa1c406f4c0 2024-04-25T16:27:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.049+0000 7fa1abfff640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa1b00744b0 0x7fa1b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.049+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa1b40950d0 con 0x7fa1c406f4c0 2024-04-25T16:27:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.049+0000 7fa1c2ffd640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa1b00744b0 0x7fa1b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.050+0000 7fa1c2ffd640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa1b00744b0 0x7fa1b00769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40b1b60 tx=0x7fa1b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.054+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1c41608a0 con 0x7fa1c406f4c0 2024-04-25T16:27:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.229+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1c40721a0 con 0x7fa1c406f4c0 2024-04-25T16:27:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.230+0000 7fa1abfff640 1 -- 172.21.15.94:0/3344031163 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fa1b405ef60 con 0x7fa1c406f4c0 2024-04-25T16:27:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:03.231 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:03.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa1b00744b0 msgr2=0x7fa1b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa1b00744b0 0x7fa1b00769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40b1b60 tx=0x7fa1b8008040 comp rx=0 tx=0).stop 2024-04-25T16:27:03.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 msgr2=0x7fa1c41f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 0x7fa1c41f9af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d620 tx=0x7fa1b400dae0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 shutdown_connections 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa1b00744b0 0x7fa1b00769c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1c40d2d70 0x7fa1c41fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c409b9b0 0x7fa1c41fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 --2- 172.21.15.94:0/3344031163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1c406f4c0 0x7fa1c41f9af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.232+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 >> 172.21.15.94:0/3344031163 conn(0x7fa1c40b7cc0 msgr2=0x7fa1c40b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.233+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 shutdown_connections 2024-04-25T16:27:03.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.233+0000 7fa1c9bb7640 1 -- 172.21.15.94:0/3344031163 wait complete. 2024-04-25T16:27:03.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T16:26:55.900210+0000', 'modified': '2024-04-25T16:27:00.941324+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6834/4275391000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 4275391000}, {'type': 'v1', 'addr': '172.21.15.138:6834', 'nonce': 4275391000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-25T16:27:03.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T16:27:03.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T16:27:03.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.755+0000 7fe1e8314640 1 Processor -- start 2024-04-25T16:27:03.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.755+0000 7fe1e8314640 1 -- start start 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e8314640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e015d750 0x7fe1e00d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e8314640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe1e00d33c0 0x7fe1e00ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e8314640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e00d37d0 con 0x7fe1e009bc70 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e8314640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00cd540 con 0x7fe1e00d33c0 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e8314640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e009c1e0 con 0x7fe1e015d750 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e68b1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe1e00d33c0 0x7fe1e00ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e68b1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe1e00d33c0 0x7fe1e00ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47202/0 (socket says 172.21.15.94:47202) 2024-04-25T16:27:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.756+0000 7fe1e68b1640 1 -- 172.21.15.94:0/2817082755 learned_addr learned my addr 172.21.15.94:0/2817082755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e58af640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e015d750 0x7fe1e00d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e68b1640 1 -- 172.21.15.94:0/2817082755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e015d750 msgr2=0x7fe1e00d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e015d750 0x7fe1e00d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e68b1640 1 -- 172.21.15.94:0/2817082755 >> v1:172.21.15.138:6789/0 conn(0x7fe1e009bc70 legacy=0x7fe1e006e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e68b1640 1 -- 172.21.15.94:0/2817082755 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e00cd790 con 0x7fe1e00d33c0 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e58af640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e015d750 0x7fe1e00d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.757+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e00d33c0 0x7fe1e00ce270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009e50 tx=0x7fe1d802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a76ca87b0150ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e50ae640 1 -- 172.21.15.94:0/2817082755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8036070 con 0x7fe1e00d33c0 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e50ae640 1 -- 172.21.15.94:0/2817082755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d8031600 con 0x7fe1e00d33c0 2024-04-25T16:27:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e50ae640 1 -- 172.21.15.94:0/2817082755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80319d0 con 0x7fe1e00d33c0 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 -- 172.21.15.94:0/2817082755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e00d33c0 msgr2=0x7fe1e00ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e00d33c0 0x7fe1e00ce270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009e50 tx=0x7fe1d802a710 comp rx=0 tx=0).stop 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 -- 172.21.15.94:0/2817082755 shutdown_connections 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e00d33c0 0x7fe1e00ce270 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 --2- 172.21.15.94:0/2817082755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e015d750 0x7fe1e00d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 -- 172.21.15.94:0/2817082755 >> 172.21.15.94:0/2817082755 conn(0x7fe1e00b7b90 msgr2=0x7fe1e00b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 -- 172.21.15.94:0/2817082755 shutdown_connections 2024-04-25T16:27:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.758+0000 7fe1e8314640 1 -- 172.21.15.94:0/2817082755 wait complete. 2024-04-25T16:27:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.760+0000 7fe1e8314640 1 Processor -- start 2024-04-25T16:27:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.760+0000 7fe1e8314640 1 -- start start 2024-04-25T16:27:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e8314640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e009bc70 0x7fe1e00c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e8314640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e00d33c0 0x7fe1e01f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e8314640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 0x7fe1e01f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e8314640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e00ce7b0 con 0x7fe1e015d750 2024-04-25T16:27:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e8314640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e00cdd50 con 0x7fe1e009bc70 2024-04-25T16:27:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e8314640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e00ce930 con 0x7fe1e00d33c0 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.761+0000 7fe1e68b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 0x7fe1e01f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 0x7fe1e01f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47206/0 (socket says 172.21.15.94:47206) 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 -- 172.21.15.94:0/3692438280 learned_addr learned my addr 172.21.15.94:0/3692438280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 -- 172.21.15.94:0/3692438280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e00d33c0 msgr2=0x7fe1e01f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e00d33c0 0x7fe1e01f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 -- 172.21.15.94:0/3692438280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e009bc70 msgr2=0x7fe1e00c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e009bc70 0x7fe1e00c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 -- 172.21.15.94:0/3692438280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e00d2430 con 0x7fe1e015d750 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 0x7fe1e01f6d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009f70 tx=0x7fe1d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1d6ffd640 1 -- 172.21.15.94:0/3692438280 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d803f020 con 0x7fe1e015d750 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1d6ffd640 1 -- 172.21.15.94:0/3692438280 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d8037040 con 0x7fe1e015d750 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e00c99d0 con 0x7fe1e015d750 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1d6ffd640 1 -- 172.21.15.94:0/3692438280 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8036070 con 0x7fe1e015d750 2024-04-25T16:27:03.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.762+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e00c9410 con 0x7fe1e015d750 2024-04-25T16:27:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.764+0000 7fe1d6ffd640 1 -- 172.21.15.94:0/3692438280 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1d8042020 con 0x7fe1e015d750 2024-04-25T16:27:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.764+0000 7fe1d6ffd640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1c40744e0 0x7fe1c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.764+0000 7fe1d6ffd640 1 -- 172.21.15.94:0/3692438280 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe1d80b59a0 con 0x7fe1e015d750 2024-04-25T16:27:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.764+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1c40744e0 0x7fe1c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.765+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1c40744e0 0x7fe1c40769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc000fb0 tx=0x7fe1dc0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.765+0000 7fe1ef1c7740 1 Processor -- start 2024-04-25T16:27:03.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.766+0000 7fe1ef1c7740 1 -- start start 2024-04-25T16:27:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f66460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557095ea9740 con 0x557095b7a3b0 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557095e453a0 con 0x557095ba3270 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557095ea5700 con 0x557095b82590 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f45260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47216/0 (socket says 172.21.15.94:47216) 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 -- 172.21.15.94:0/1031848805 learned_addr learned my addr 172.21.15.94:0/1031848805 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f66460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 -- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 msgr2=0x557095f457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 -- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 msgr2=0x557095f66460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f66460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.767+0000 7fe1e58af640 1 -- 172.21.15.94:0/1031848805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557095e61750 con 0x557095b7a3b0 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.768+0000 7fe1e58af640 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f45260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0007720 tx=0x7fe1d0007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783d1875852ef60b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.768+0000 7fe1c3fff640 1 -- 172.21.15.94:0/1031848805 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0007e30 con 0x557095b7a3b0 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.768+0000 7fe1c3fff640 1 -- 172.21.15.94:0/1031848805 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d0004030 con 0x557095b7a3b0 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.768+0000 7fe1c3fff640 1 -- 172.21.15.94:0/1031848805 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00043e0 con 0x557095b7a3b0 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/1031848805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 msgr2=0x557095f45260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f45260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0007720 tx=0x7fe1d0007be0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/1031848805 shutdown_connections 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f45260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/1031848805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f66460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/1031848805 >> 172.21.15.94:0/1031848805 conn(0x557095b973c0 msgr2=0x557095f449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/1031848805 shutdown_connections 2024-04-25T16:27:03.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.769+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/1031848805 wait complete. 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.770+0000 7fe1ef1c7740 1 -- start start 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.770+0000 7fe1ef1c7740 1 Processor -- start 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f68b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557095e30fe0 con 0x557095b82590 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557095f87cd0 con 0x557095ba3270 2024-04-25T16:27:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557095f686e0 con 0x557095b7a3b0 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1ef1c7740 10 client.0 _ll_register_callbacks cb 0x557095e47150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47232/0 (socket says 172.21.15.94:47232) 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 -- 172.21.15.94:0/2105733146 learned_addr learned my addr 172.21.15.94:0/2105733146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f68b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 -- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 msgr2=0x557095f657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 -- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 msgr2=0x557095f68b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f68b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e58af640 1 -- 172.21.15.94:0/2105733146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557095bbb2d0 con 0x557095b82590 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.771+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f68b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1e58af640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6cb90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0007010 tx=0x7fe1d0006df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b51b7b3f6a2802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1c17fa640 1 -- 172.21.15.94:0/2105733146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000db50 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1c17fa640 1 -- 172.21.15.94:0/2105733146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe1d000dcf0 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d0004030 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1c17fa640 1 -- 172.21.15.94:0/2105733146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fe1d002c050 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x557095c856b0 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1c17fa640 1 -- 172.21.15.94:0/2105733146 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d0004030 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1c17fa640 1 -- 172.21.15.94:0/2105733146 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fe1d002f020 con 0x557095b82590 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 20 client.4401 populate_metadata read hostname 'smithi094' 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095b61e40 0x557095fa9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 4 client.4401 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.772+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 --> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557095eb7350 con 0x557095b61e40 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.773+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095b61e40 0x557095fa9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.773+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095b61e40 0x557095fa9420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f213e45b279e5dc0 server_cookie=f038c9157f53e942 in_seq=0 out_seq=0 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.773+0000 7fe1c17fa640 10 client.4401 ms_handle_connect on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1c17fa640 1 -- 172.21.15.94:0/2105733146 <== mds.0 v2:172.21.15.138:6832/4275391000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557095eb7350 con 0x557095b61e40 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1c17fa640 10 client.4401 handle_command_reply: tid=0 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 client.4401 shutdown 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 msgr2=0x557095f6cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6cb90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0007010 tx=0x7fe1d0006df0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 shutdown_connections 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f68b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095b61e40 0x557095fa9420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6cb90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/2105733146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1c17fa640 0 client.4401 ms_handle_reset on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:03.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 >> 172.21.15.94:0/2105733146 conn(0x557095b973c0 msgr2=0x557095f3ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.774+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 shutdown_connections 2024-04-25T16:27:03.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.775+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/2105733146 wait complete. 2024-04-25T16:27:03.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.775+0000 7fe1ef1c7740 20 client.4401 trim_cache size 0 max 16384 2024-04-25T16:27:03.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.793+0000 7fe1ef1c7740 1 Processor -- start 2024-04-25T16:27:03.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.793+0000 7fe1ef1c7740 1 -- start start 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.794+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f6c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.794+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f5e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557096002200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e006dce0 con 0x557095b7a3b0 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0150040 con 0x557095ba3270 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e006d690 con 0x557095b82590 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f6c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f6c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47234/0 (socket says 172.21.15.94:47234) 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 -- 172.21.15.94:0/994101680 learned_addr learned my addr 172.21.15.94:0/994101680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 -- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 msgr2=0x557095f5e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 -- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 msgr2=0x557096002200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557096002200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557096002200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 -- 172.21.15.94:0/994101680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557095f6cf00 con 0x557095b7a3b0 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f6c040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0074040 tx=0x7fe1dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6d0429c0a67a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1c37fe640 1 -- 172.21.15.94:0/994101680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc014930 con 0x557095b7a3b0 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1c37fe640 1 -- 172.21.15.94:0/994101680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1dc014ad0 con 0x557095b7a3b0 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.795+0000 7fe1c37fe640 1 -- 172.21.15.94:0/994101680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0250d0 con 0x557095b7a3b0 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/994101680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 msgr2=0x557095f6c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f6c040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0074040 tx=0x7fe1dc012000 comp rx=0 tx=0).stop 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/994101680 shutdown_connections 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557096002200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b82590 0x557095f5e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/994101680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b7a3b0 0x557095f6c040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/994101680 >> 172.21.15.94:0/994101680 conn(0x557095b973c0 msgr2=0x557095f724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/994101680 shutdown_connections 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.796+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/994101680 wait complete. 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.797+0000 7fe1ef1c7740 1 -- start start 2024-04-25T16:27:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.797+0000 7fe1ef1c7740 1 Processor -- start 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557095f69500 con 0x557095b82590 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557095f690b0 con 0x557095ba3270 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557095f5e650 con 0x557095b7a3b0 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1ef1c7740 10 client.0 _ll_register_callbacks cb 0x7fe1dc001210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1e58af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1e58af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47238/0 (socket says 172.21.15.94:47238) 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1e58af640 1 -- 172.21.15.94:0/978003785 learned_addr learned my addr 172.21.15.94:0/978003785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.798+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e68b1640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e58af640 1 -- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 msgr2=0x557095f6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e58af640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f6d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e58af640 1 -- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 msgr2=0x557095f449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e58af640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e58af640 1 -- 172.21.15.94:0/978003785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557095fa3a50 con 0x557095b82590 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1e58af640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6d970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00045c0 tx=0x7fe1d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a66ccc19c87bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1c17fa640 1 -- 172.21.15.94:0/978003785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0007570 con 0x557095b82590 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1c17fa640 1 -- 172.21.15.94:0/978003785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe1d0003eb0 con 0x557095b82590 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1dc014ad0 con 0x557095b82590 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1c17fa640 1 -- 172.21.15.94:0/978003785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fe1d00040b0 con 0x557095b82590 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 10 client.4407 fetch_fsmap learned FSMap version 7 2024-04-25T16:27:03.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x557095f733a0 con 0x557095b82590 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1c17fa640 1 -- 172.21.15.94:0/978003785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1dc014ad0 con 0x557095b82590 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1c17fa640 1 -- 172.21.15.94:0/978003785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fe1d002f070 con 0x557095b82590 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 10 client.4407 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 20 client.4407 populate_metadata read hostname 'smithi094' 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095f43210 0x557095f43620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 4 client.4407 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.799+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 --> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] -- command(tid 0: {"prefix": "status"}) -- 0x557095eb11c0 con 0x557095f43210 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095f43210 0x557095f43620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1c17fa640 10 client.4407 ms_handle_connect on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1e60b0640 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095f43210 0x557095f43620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10a3eb58b105777 server_cookie=df10e844a1c1e83f in_seq=0 out_seq=0 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1c17fa640 1 -- 172.21.15.94:0/978003785 <== mds.0 v2:172.21.15.138:6832/4275391000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x557095eb11c0 con 0x557095f43210 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1c17fa640 10 client.4407 handle_command_reply: tid=0 2024-04-25T16:27:03.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 client.4407 shutdown 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 msgr2=0x557095f6d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6d970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00045c0 tx=0x7fe1d00046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 shutdown_connections 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557095ba3270 0x557095f449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557095f43210 0x557095f43620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.800+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557095b82590 0x557095f6d970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1ef1c7740 1 --2- 172.21.15.94:0/978003785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557095b7a3b0 0x557095f6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1c17fa640 0 client.4407 ms_handle_reset on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 >> 172.21.15.94:0/978003785 conn(0x557095b973c0 msgr2=0x557095f724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 shutdown_connections 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1ef1c7740 1 -- 172.21.15.94:0/978003785 wait complete. 2024-04-25T16:27:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1ef1c7740 20 client.4407 trim_cache size 0 max 16384 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "e3c627c0-5913-4b5c-b1aa-d445103b794b", 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4279, 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 7.2644907319999996, 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 7, 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 25, 2024-04-25T16:27:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T16:27:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 14.738430745 2024-04-25T16:27:03.808 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:27:03.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1c40744e0 msgr2=0x7fe1c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1c40744e0 0x7fe1c40769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc000fb0 tx=0x7fe1dc0079c0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 msgr2=0x7fe1e01f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 0x7fe1e01f6d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009f70 tx=0x7fe1d8004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 shutdown_connections 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1c40744e0 0x7fe1c40769f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e015d750 0x7fe1e01f6d20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1e00d33c0 0x7fe1e01f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 --2- 172.21.15.94:0/3692438280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e009bc70 0x7fe1e00c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 >> 172.21.15.94:0/3692438280 conn(0x7fe1e00b7b90 msgr2=0x7fe1e015bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 shutdown_connections 2024-04-25T16:27:03.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.801+0000 7fe1e8314640 1 -- 172.21.15.94:0/3692438280 wait complete. 2024-04-25T16:27:03.813 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:27:03.813 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:03.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:27:03.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.920+0000 7fcf69940640 1 Processor -- start 2024-04-25T16:27:03.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.920+0000 7fcf69940640 1 -- start start 2024-04-25T16:27:03.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.921+0000 7fcf69940640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 0x7fcf6406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.921+0000 7fcf69940640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6406c9c0 con 0x7fcf6408e780 2024-04-25T16:27:03.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.921+0000 7fcf69940640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6406cca0 con 0x7fcf640d1010 2024-04-25T16:27:03.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.921+0000 7fcf69940640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6406dfa0 con 0x7fcf640d4bb0 2024-04-25T16:27:03.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf62ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fcf6408e780 0x7fcf640b5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53062/0 (socket says 172.21.15.94:53062) 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf62ffd640 1 -- 172.21.15.94:0/2683373198 learned_addr learned my addr 172.21.15.94:0/2683373198 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162538498 0 0) 0x7fcf6406c9c0 con 0x7fcf6408e780 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf4c003520 con 0x7fcf6408e780 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf627fc640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 0x7fcf6406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3634584896 0 0) 0x7fcf4c003520 con 0x7fcf6408e780 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 msgr2=0x7fcf6406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 0x7fcf6406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 >> v1:172.21.15.138:6789/0 conn(0x7fcf640d1010 legacy=0x7fcf640d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6406cf80 con 0x7fcf6408e780 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fcf54001670 con 0x7fcf6408e780 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 >> v1:172.21.15.94:6789/0 conn(0x7fcf6408e780 legacy=0x7fcf640b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 0x7fcf4c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4c003f40 0x7fcf4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c004d40 0x7fcf4c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6406dfa0 con 0x7fcf4c003f40 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c00bad0 con 0x7fcf4c004d40 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c00bcd0 con 0x7fcf640d4bb0 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf627fc640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4c003f40 0x7fcf4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf62ffd640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 0x7fcf4c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.922+0000 7fcf637fe640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c004d40 0x7fcf4c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf627fc640 1 -- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 msgr2=0x7fcf4c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf627fc640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf640d4bb0 0x7fcf4c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf627fc640 1 -- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c004d40 msgr2=0x7fcf4c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf627fc640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c004d40 0x7fcf4c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf627fc640 1 -- 172.21.15.94:0/2683373198 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c003520 con 0x7fcf4c003f40 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf627fc640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4c003f40 0x7fcf4c004800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009ac0 tx=0x7fcf4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6d575ee1513c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58036070 con 0x7fcf4c003f40 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58031400 con 0x7fcf4c003f40 2024-04-25T16:27:03.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.923+0000 7fcf61ffb640 1 -- 172.21.15.94:0/2683373198 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009690 con 0x7fcf4c003f40 2024-04-25T16:27:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf69940640 1 -- 172.21.15.94:0/2683373198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4c003f40 msgr2=0x7fcf4c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf69940640 1 --2- 172.21.15.94:0/2683373198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4c003f40 0x7fcf4c004800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009ac0 tx=0x7fcf4c005830 comp rx=0 tx=0).stop 2024-04-25T16:27:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf637fe640 1 -- 172.21.15.94:0/2683373198 reap_dead start 2024-04-25T16:27:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf69940640 1 -- 172.21.15.94:0/2683373198 shutdown_connections 2024-04-25T16:27:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf69940640 1 -- 172.21.15.94:0/2683373198 >> 172.21.15.94:0/2683373198 conn(0x7fcf640b7cc0 msgr2=0x7fcf640b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf69940640 1 -- 172.21.15.94:0/2683373198 shutdown_connections 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.924+0000 7fcf69940640 1 -- 172.21.15.94:0/2683373198 wait complete. 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.926+0000 7fcf69940640 1 Processor -- start 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.926+0000 7fcf69940640 1 -- start start 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf69940640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c003f40 0x7fcf640adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf69940640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 0x7fcf64150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf69940640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf641513e0 0x7fcf641517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf69940640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6406c570 con 0x7fcf640ae160 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf69940640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6406d540 con 0x7fcf4c003f40 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf69940640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6406c6f0 con 0x7fcf641513e0 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf627fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 0x7fcf64150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.927+0000 7fcf627fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 0x7fcf64150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47248/0 (socket says 172.21.15.94:47248) 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 -- 172.21.15.94:0/1242927667 learned_addr learned my addr 172.21.15.94:0/1242927667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf62ffd640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c003f40 0x7fcf640adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf637fe640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf641513e0 0x7fcf641517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 -- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf641513e0 msgr2=0x7fcf641517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf641513e0 0x7fcf641517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 -- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c003f40 msgr2=0x7fcf640adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c003f40 0x7fcf640adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 -- 172.21.15.94:0/1242927667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf640d0380 con 0x7fcf640ae160 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf627fc640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 0x7fcf64150ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009be0 tx=0x7fcf58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58040020 con 0x7fcf640ae160 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58003a40 con 0x7fcf640ae160 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf640ae9e0 con 0x7fcf640ae160 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58036040 con 0x7fcf640ae160 2024-04-25T16:27:03.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.928+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6414f340 con 0x7fcf640ae160 2024-04-25T16:27:03.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.929+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6406a180 con 0x7fcf640ae160 2024-04-25T16:27:03.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.930+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf58043020 con 0x7fcf640ae160 2024-04-25T16:27:03.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.930+0000 7fcf4bfff640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcf500744b0 0x7fcf500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:03.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.930+0000 7fcf62ffd640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcf500744b0 0x7fcf500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:03.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.930+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf580b55b0 con 0x7fcf640ae160 2024-04-25T16:27:03.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.931+0000 7fcf62ffd640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcf500744b0 0x7fcf500769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf540011e0 tx=0x7fcf54001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:03.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:03.935+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf5804a070 con 0x7fcf640ae160 2024-04-25T16:27:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.150+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf640ae770 con 0x7fcf640ae160 2024-04-25T16:27:04.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.150+0000 7fcf4bfff640 1 -- 172.21.15.94:0/1242927667 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fcf5807fb20 con 0x7fcf640ae160 2024-04-25T16:27:04.151 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcf500744b0 msgr2=0x7fcf500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcf500744b0 0x7fcf500769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf540011e0 tx=0x7fcf54001150 comp rx=0 tx=0).stop 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 msgr2=0x7fcf64150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 0x7fcf64150ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009be0 tx=0x7fcf58004730 comp rx=0 tx=0).stop 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 shutdown_connections 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcf500744b0 0x7fcf500769c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf641513e0 0x7fcf641517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf640ae160 0x7fcf64150ea0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:04.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 --2- 172.21.15.94:0/1242927667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4c003f40 0x7fcf640adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:04.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 >> 172.21.15.94:0/1242927667 conn(0x7fcf640b7cc0 msgr2=0x7fcf641643a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:04.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 shutdown_connections 2024-04-25T16:27:04.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04.152+0000 7fcf69940640 1 -- 172.21.15.94:0/1242927667 wait complete. 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:27:04.164 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T16:27:04.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:04.165 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:04.179 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42709sec preferred_lft 42709sec 2024-04-25T16:27:04.180 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::55c9:4b97:ccb7:f8e8/64 scope link noprefixroute 2024-04-25T16:27:04.181 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:04.181 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:27:04.181 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:04.181 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:04.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:27:04.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-25T16:27:04.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-25T16:27:04.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:27:04.182 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:04.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:04.356 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:27:04.380 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:27:04.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:04.382 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:27:04.445 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:04.445 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:04.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:27:04.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:27:04.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:27:04.445 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:04.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:04.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:04.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:04.738 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:04.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:04.753 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T16:27:04.807 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:04.807 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:04.807 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:04.807 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T16:27:04.807 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:04.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:04.926 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T16:27:04.926 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:04.926 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:04.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T16:27:04.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:27:04.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T16:27:04.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T16:27:04.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T16:27:04.927 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:04.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:05.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:05.053 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:05.053 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:05.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T16:27:05.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:27:05.053 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:05.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:05.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:05.127 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T16:27:05.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:05.127 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:05.141 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T16:27:05.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:05.142 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:05.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:27:05.261 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:27:05.317 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:27:05.318 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:05.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:05.319 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:27:05.371 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T16:27:05.372 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T16:27:05.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:27:05.437 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:27:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:27:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:27:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:27:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:27:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:27:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:27:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:05.494 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:05.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:05.495 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:27:05.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:2024-04-25T16:27:05.554+0000 7fb20f78d480 -1 init, newargv = 0x55a3971d42f0 newargc=15 2024-04-25T16:27:05.555 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[41657]: starting ceph client 2024-04-25T16:27:05.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[41657]: starting fuse 2024-04-25T16:27:06.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:27:06.676 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:27:06.732 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:27:06.732 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:27:06.732 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:27:06.732 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:27:06.733 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:06.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:06.735 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:27:06.791 INFO:teuthology.orchestra.run.smithi094.stdout:43 2024-04-25T16:27:06.791 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T16:27:06.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:27:06.791 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-04-25T16:27:06.791 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-04-25T16:27:06.791 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-04-25T16:27:06.791 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:27:06.792 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.admin")) 2024-04-25T16:27:06.793 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-04-25T16:27:06.911 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.admin.41657.asok 2024-04-25T16:27:06.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:06.922 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41657.asok 2024-04-25T16:27:06.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:06.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41657.asok status 2024-04-25T16:27:07.061 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:27:07.061 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "admin", 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "41657", 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4416, 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-04-25T16:27:07.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4416 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 193775313 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 193775313 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4416 192.168.0.1:0/193775313", 2024-04-25T16:27:07.063 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/193775313", 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 7, 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 25, 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-04-25T16:27:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:27:07.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:07.070 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-04-25T16:27:07.070 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:07.085 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-04-25T16:27:07.085 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:07.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:07.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:07.149 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T16:27:07.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:07.255 INFO:teuthology.orchestra.run.smithi094.stdout:# file: . 2024-04-25T16:27:07.255 INFO:teuthology.orchestra.run.smithi094.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T16:27:07.255 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:07.261 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:07.261 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:07.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.422+0000 7f86099c7640 1 Processor -- start 2024-04-25T16:27:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.422+0000 7f86099c7640 1 -- start start 2024-04-25T16:27:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86099c7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f860408e780 0x7f860406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86099c7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f86040d4bb0 0x7f86040b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86099c7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86040b5880 con 0x7f86040d1010 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86099c7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f860406dfa0 con 0x7f86040d4bb0 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86099c7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f860406c9c0 con 0x7f860408e780 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86027fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f86040d4bb0 0x7f86040b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86037fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f86040d1010 0x7f86040d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53068/0 (socket says 172.21.15.94:53068) 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86027fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f86040d4bb0 0x7f86040b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47254/0 (socket says 172.21.15.94:47254) 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86037fe640 1 -- 172.21.15.94:0/3801558798 learned_addr learned my addr 172.21.15.94:0/3801558798 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f8602ffd640 1 --2- 172.21.15.94:0/3801558798 >> v2:172.21.15.138:3300/0 conn(0x7f860408e780 0x7f860406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86027fc640 1 -- 172.21.15.94:0/3801558798 >> v2:172.21.15.138:3300/0 conn(0x7f860408e780 msgr2=0x7f860406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86027fc640 1 --2- 172.21.15.94:0/3801558798 >> v2:172.21.15.138:3300/0 conn(0x7f860408e780 0x7f860406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86027fc640 1 -- 172.21.15.94:0/3801558798 >> v1:172.21.15.94:6789/0 conn(0x7f86040d1010 legacy=0x7f86040d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f86027fc640 1 -- 172.21.15.94:0/3801558798 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860406cca0 con 0x7f86040d4bb0 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.424+0000 7f8601ffb640 1 -- 172.21.15.94:0/3801558798 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487994219 0 0) 0x7f86040b5880 con 0x7f86040d1010 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.425+0000 7f86027fc640 1 --2- 172.21.15.94:0/3801558798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86040d4bb0 0x7f86040b5340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f860408d6e0 tx=0x7f85ec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21be80c23d2f189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.425+0000 7f8601ffb640 1 -- 172.21.15.94:0/3801558798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec037070 con 0x7f86040d4bb0 2024-04-25T16:27:07.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.425+0000 7f8601ffb640 1 -- 172.21.15.94:0/3801558798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec032440 con 0x7f86040d4bb0 2024-04-25T16:27:07.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.425+0000 7f8601ffb640 1 -- 172.21.15.94:0/3801558798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec02ae60 con 0x7f86040d4bb0 2024-04-25T16:27:07.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 -- 172.21.15.94:0/3801558798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86040d4bb0 msgr2=0x7f86040b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 --2- 172.21.15.94:0/3801558798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86040d4bb0 0x7f86040b5340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f860408d6e0 tx=0x7f85ec02a110 comp rx=0 tx=0).stop 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 -- 172.21.15.94:0/3801558798 shutdown_connections 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 --2- 172.21.15.94:0/3801558798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86040d4bb0 0x7f86040b5340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 --2- 172.21.15.94:0/3801558798 >> v2:172.21.15.138:3300/0 conn(0x7f860408e780 0x7f860406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 -- 172.21.15.94:0/3801558798 >> 172.21.15.94:0/3801558798 conn(0x7f86040b7cc0 msgr2=0x7f86040b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 -- 172.21.15.94:0/3801558798 shutdown_connections 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.426+0000 7f86099c7640 1 -- 172.21.15.94:0/3801558798 wait complete. 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.427+0000 7f86099c7640 1 Processor -- start 2024-04-25T16:27:07.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.428+0000 7f86099c7640 1 -- start start 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86099c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 0x7f86041607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86099c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86040d1010 0x7f860414e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86099c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86040d4bb0 0x7f86040cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86099c7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f860406d540 con 0x7f860408e780 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86099c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f860406d260 con 0x7f86040d1010 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86099c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f860406c570 con 0x7f86040d4bb0 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 0x7f86041607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 0x7f86041607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47270/0 (socket says 172.21.15.94:47270) 2024-04-25T16:27:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 -- 172.21.15.94:0/299224514 learned_addr learned my addr 172.21.15.94:0/299224514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86027fc640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86040d1010 0x7f860414e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f86037fe640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86040d4bb0 0x7f86040cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 -- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86040d4bb0 msgr2=0x7f86040cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86040d4bb0 0x7f86040cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 -- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86040d1010 msgr2=0x7f860414e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86040d1010 0x7f860414e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 -- 172.21.15.94:0/299224514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8604161130 con 0x7f860408e780 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.429+0000 7f8602ffd640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 0x7f86041607c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85fc007e40 tx=0x7f85fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.430+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc014620 con 0x7f860408e780 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.430+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85fc019070 con 0x7f860408e780 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.430+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8604161410 con 0x7f860408e780 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.430+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc014a00 con 0x7f860408e780 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.430+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041619d0 con 0x7f860408e780 2024-04-25T16:27:07.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.431+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860406a180 con 0x7f860408e780 2024-04-25T16:27:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.431+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85fc014ba0 con 0x7f860408e780 2024-04-25T16:27:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.432+0000 7f85fbfff640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f85e40744b0 0x7f85e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.432+0000 7f86027fc640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f85e40744b0 0x7f85e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.432+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f85fc095d50 con 0x7f860408e780 2024-04-25T16:27:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.432+0000 7f86027fc640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f85e40744b0 0x7f85e40769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85ec034000 tx=0x7f85ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:07.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.437+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85fc099030 con 0x7f860408e780 2024-04-25T16:27:07.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.613+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86040b8590 con 0x7f860408e780 2024-04-25T16:27:07.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.614+0000 7f85fbfff640 1 -- 172.21.15.94:0/299224514 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f85fc060230 con 0x7f860408e780 2024-04-25T16:27:07.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:07.615 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f85e40744b0 msgr2=0x7f85e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f85e40744b0 0x7f85e40769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85ec034000 tx=0x7f85ec02e000 comp rx=0 tx=0).stop 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 msgr2=0x7f86041607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 0x7f86041607c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85fc007e40 tx=0x7f85fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 shutdown_connections 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f85e40744b0 0x7f85e40769c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86040d4bb0 0x7f86040cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86040d1010 0x7f860414e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 --2- 172.21.15.94:0/299224514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860408e780 0x7f86041607c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 >> 172.21.15.94:0/299224514 conn(0x7f86040b7cc0 msgr2=0x7f86040bb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 shutdown_connections 2024-04-25T16:27:07.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.616+0000 7f86099c7640 1 -- 172.21.15.94:0/299224514 wait complete. 2024-04-25T16:27:07.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:07.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:07.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.739+0000 7f2be9584640 1 Processor -- start 2024-04-25T16:27:07.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.739+0000 7f2be9584640 1 -- start start 2024-04-25T16:27:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be9584640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 0x7f2be4160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be9584640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2be40d4a20 0x7f2be40d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be9584640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be4161930 con 0x7f2be40b5710 2024-04-25T16:27:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be9584640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2be4161c10 con 0x7f2be40d4a20 2024-04-25T16:27:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be9584640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2be4164d10 con 0x7f2be40b4c70 2024-04-25T16:27:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be37fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2be40d4a20 0x7f2be40d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be27fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 0x7f2be4160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be37fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2be40d4a20 0x7f2be40d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43932/0 (socket says 172.21.15.94:43932) 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be27fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 0x7f2be4160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:59260/0 (socket says 172.21.15.94:59260) 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be37fe640 1 -- 172.21.15.94:0/682319502 learned_addr learned my addr 172.21.15.94:0/682319502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.740+0000 7f2be2ffd640 1 --1- 172.21.15.94:0/682319502 >> v1:172.21.15.138:6789/0 conn(0x7f2be40b5710 0x7f2be41608e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:36958/0 (socket says 172.21.15.94:36958) 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be37fe640 1 -- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 msgr2=0x7f2be4160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be37fe640 1 --2- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 0x7f2be4160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be37fe640 1 -- 172.21.15.94:0/682319502 >> v1:172.21.15.138:6789/0 conn(0x7f2be40b5710 legacy=0x7f2be41608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be1ffb640 1 -- 172.21.15.94:0/682319502 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344727900 0 0) 0x7f2be4161930 con 0x7f2be40b5710 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be37fe640 1 -- 172.21.15.94:0/682319502 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4164ff0 con 0x7f2be40d4a20 2024-04-25T16:27:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be27fc640 1 --2- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 0x7f2be4160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.741+0000 7f2be37fe640 1 --2- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40d4a20 0x7f2be40d4e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a44fc47650ae65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be1ffb640 1 -- 172.21.15.94:0/682319502 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc037070 con 0x7f2be40d4a20 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be1ffb640 1 -- 172.21.15.94:0/682319502 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bdc032440 con 0x7f2be40d4a20 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be1ffb640 1 -- 172.21.15.94:0/682319502 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc032810 con 0x7f2be40d4a20 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 -- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40d4a20 msgr2=0x7f2be40d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 --2- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40d4a20 0x7f2be40d4e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc02a130 comp rx=0 tx=0).stop 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 -- 172.21.15.94:0/682319502 shutdown_connections 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 --2- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40d4a20 0x7f2be40d4e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 --2- 172.21.15.94:0/682319502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b4c70 0x7f2be4160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 -- 172.21.15.94:0/682319502 >> 172.21.15.94:0/682319502 conn(0x7f2be40d6930 msgr2=0x7f2be40b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 -- 172.21.15.94:0/682319502 shutdown_connections 2024-04-25T16:27:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.742+0000 7f2be9584640 1 -- 172.21.15.94:0/682319502 wait complete. 2024-04-25T16:27:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.744+0000 7f2be9584640 1 Processor -- start 2024-04-25T16:27:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.744+0000 7f2be9584640 1 -- start start 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be9584640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40b4c70 0x7f2be4159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be9584640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b5710 0x7f2be40cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be9584640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 0x7f2be40cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be9584640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4161530 con 0x7f2be40d4a20 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be9584640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2be40b51c0 con 0x7f2be40b4c70 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be9584640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2be41616b0 con 0x7f2be40b5710 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 0x7f2be40cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 0x7f2be40cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47278/0 (socket says 172.21.15.94:47278) 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 -- 172.21.15.94:0/3371957590 learned_addr learned my addr 172.21.15.94:0/3371957590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be2ffd640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40b4c70 0x7f2be4159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be27fc640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b5710 0x7f2be40cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 -- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b5710 msgr2=0x7f2be40cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b5710 0x7f2be40cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 -- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40b4c70 msgr2=0x7f2be4159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40b4c70 0x7f2be4159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 -- 172.21.15.94:0/3371957590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4162e40 con 0x7f2be40d4a20 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2be37fe640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 0x7f2be40cd720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0090b0 tx=0x7f2bdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.746+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc037040 con 0x7f2be40d4a20 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.747+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bdc004230 con 0x7f2be40d4a20 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.747+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be40cdc60 con 0x7f2be40d4a20 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.747+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003610 con 0x7f2be40d4a20 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.747+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be415b450 con 0x7f2be40d4a20 2024-04-25T16:27:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.748+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be41630b0 con 0x7f2be40d4a20 2024-04-25T16:27:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.748+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bdc0037b0 con 0x7f2be40d4a20 2024-04-25T16:27:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.749+0000 7f2bcb7fe640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2bd00744e0 0x7f2bd00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.749+0000 7f2be2ffd640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2bd00744e0 0x7f2bd00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.749+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2bdc0b6140 con 0x7f2be40d4a20 2024-04-25T16:27:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.749+0000 7f2be2ffd640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2bd00744e0 0x7f2bd00769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40012e0 tx=0x7f2bd4001630 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:07.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.753+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bdc07ffd0 con 0x7f2be40d4a20 2024-04-25T16:27:07.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.926+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2be406a180 con 0x7f2be40d4a20 2024-04-25T16:27:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.926+0000 7f2bcb7fe640 1 -- 172.21.15.94:0/3371957590 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f2bdc0842d0 con 0x7f2be40d4a20 2024-04-25T16:27:07.927 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2bd00744e0 msgr2=0x7f2bd00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2bd00744e0 0x7f2bd00769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40012e0 tx=0x7f2bd4001630 comp rx=0 tx=0).stop 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 msgr2=0x7f2be40cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 0x7f2be40cd720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0090b0 tx=0x7f2bdc0046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 shutdown_connections 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2bd00744e0 0x7f2bd00769f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be40d4a20 0x7f2be40cd720 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2be40b5710 0x7f2be40cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 --2- 172.21.15.94:0/3371957590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be40b4c70 0x7f2be4159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:07.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 >> 172.21.15.94:0/3371957590 conn(0x7f2be40d6930 msgr2=0x7f2be406ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:07.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 shutdown_connections 2024-04-25T16:27:07.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:07.928+0000 7f2be9584640 1 -- 172.21.15.94:0/3371957590 wait complete. 2024-04-25T16:27:07.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.049+0000 7fbb02164640 1 Processor -- start 2024-04-25T16:27:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.049+0000 7fbb02164640 1 -- start start 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.050+0000 7fbb02164640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbafc0b24b0 0x7fbafc157940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.050+0000 7fbb02164640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbafc1619b0 0x7fbafc0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.050+0000 7fbb02164640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc0b35b0 con 0x7fbafc0d2ba0 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.050+0000 7fbb02164640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbafc0b2ce0 con 0x7fbafc0b24b0 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbb02164640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbafc0b0990 con 0x7fbafc1619b0 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbafc0b24b0 0x7fbafc157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbafc0b24b0 0x7fbafc157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47290/0 (socket says 172.21.15.94:47290) 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2866091070 learned_addr learned my addr 172.21.15.94:0/2866091070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafaffd640 1 --2- 172.21.15.94:0/2866091070 >> v2:172.21.15.138:3300/0 conn(0x7fbafc1619b0 0x7fbafc0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2866091070 >> v2:172.21.15.138:3300/0 conn(0x7fbafc1619b0 msgr2=0x7fbafc0b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 --2- 172.21.15.94:0/2866091070 >> v2:172.21.15.138:3300/0 conn(0x7fbafc1619b0 0x7fbafc0b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2866091070 >> v1:172.21.15.138:6789/0 conn(0x7fbafc0d2ba0 legacy=0x7fbafc0d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.051+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2866091070 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc0b2a00 con 0x7fbafc0b24b0 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.052+0000 7fbafb7fe640 1 --2- 172.21.15.94:0/2866091070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc157940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbafc06fad0 tx=0x7fbaec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac443f6eb85d72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.052+0000 7fbafa7fc640 1 -- 172.21.15.94:0/2866091070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec036070 con 0x7fbafc0b24b0 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.052+0000 7fbafa7fc640 1 -- 172.21.15.94:0/2866091070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaec031600 con 0x7fbafc0b24b0 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.052+0000 7fbafa7fc640 1 -- 172.21.15.94:0/2866091070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0319d0 con 0x7fbafc0b24b0 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.052+0000 7fbb02164640 1 -- 172.21.15.94:0/2866091070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 msgr2=0x7fbafc157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.052+0000 7fbb02164640 1 --2- 172.21.15.94:0/2866091070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc157940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbafc06fad0 tx=0x7fbaec02a6f0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.053+0000 7fbb02164640 1 -- 172.21.15.94:0/2866091070 shutdown_connections 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.053+0000 7fbb02164640 1 --2- 172.21.15.94:0/2866091070 >> v2:172.21.15.138:3300/0 conn(0x7fbafc1619b0 0x7fbafc0b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.053+0000 7fbb02164640 1 --2- 172.21.15.94:0/2866091070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc157940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.053+0000 7fbb02164640 1 -- 172.21.15.94:0/2866091070 >> 172.21.15.94:0/2866091070 conn(0x7fbafc0b7cc0 msgr2=0x7fbafc0b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.053+0000 7fbb02164640 1 -- 172.21.15.94:0/2866091070 shutdown_connections 2024-04-25T16:27:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.053+0000 7fbb02164640 1 -- 172.21.15.94:0/2866091070 wait complete. 2024-04-25T16:27:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.054+0000 7fbb02164640 1 Processor -- start 2024-04-25T16:27:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.054+0000 7fbb02164640 1 -- start start 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbb02164640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbb02164640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbafc0d2ba0 0x7fbafc15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbb02164640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbafc1619b0 0x7fbafc06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbb02164640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc161dc0 con 0x7fbafc0b24b0 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbb02164640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc06f620 con 0x7fbafc1619b0 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbb02164640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc0b0f10 con 0x7fbafc0d2ba0 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47300/0 (socket says 172.21.15.94:47300) 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2310690234 learned_addr learned my addr 172.21.15.94:0/2310690234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafbfff640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbafc1619b0 0x7fbafc06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafaffd640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbafc0d2ba0 0x7fbafc15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbafc0d2ba0 msgr2=0x7fbafc15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbafc0d2ba0 0x7fbafc15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbafc1619b0 msgr2=0x7fbafc06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbafc1619b0 0x7fbafc06e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 -- 172.21.15.94:0/2310690234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc1486a0 con 0x7fbafc0b24b0 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbafb7fe640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc148160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbaec034040 tx=0x7fbaec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec036050 con 0x7fbafc0b24b0 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.056+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaec042070 con 0x7fbafc0b24b0 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.057+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc06d050 con 0x7fbafc0b24b0 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.057+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec009510 con 0x7fbafc0b24b0 2024-04-25T16:27:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.057+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc06d610 con 0x7fbafc0b24b0 2024-04-25T16:27:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.058+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc06a180 con 0x7fbafc0b24b0 2024-04-25T16:27:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.058+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbaec007c50 con 0x7fbafc0b24b0 2024-04-25T16:27:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.058+0000 7fbaf3fff640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbadc0744e0 0x7fbadc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.059+0000 7fbafaffd640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbadc0744e0 0x7fbadc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.059+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbaec0b6540 con 0x7fbafc0b24b0 2024-04-25T16:27:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.059+0000 7fbafaffd640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbadc0744e0 0x7fbadc0769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006fd0 tx=0x7fbae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.063+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbaec080990 con 0x7fbafc0b24b0 2024-04-25T16:27:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.239+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbafc06cd80 con 0x7fbafc0b24b0 2024-04-25T16:27:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.239+0000 7fbaf3fff640 1 -- 172.21.15.94:0/2310690234 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fbaec080290 con 0x7fbafc0b24b0 2024-04-25T16:27:08.240 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbadc0744e0 msgr2=0x7fbadc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbadc0744e0 0x7fbadc0769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006fd0 tx=0x7fbae4008040 comp rx=0 tx=0).stop 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 msgr2=0x7fbafc148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc148160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbaec034040 tx=0x7fbaec004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 shutdown_connections 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbadc0744e0 0x7fbadc0769f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbafc1619b0 0x7fbafc06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbafc0d2ba0 0x7fbafc15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 --2- 172.21.15.94:0/2310690234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbafc0b24b0 0x7fbafc148160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 >> 172.21.15.94:0/2310690234 conn(0x7fbafc0b7cc0 msgr2=0x7fbafc0b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 shutdown_connections 2024-04-25T16:27:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.241+0000 7fbb02164640 1 -- 172.21.15.94:0/2310690234 wait complete. 2024-04-25T16:27:08.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.362+0000 7f840cb5e640 1 Processor -- start 2024-04-25T16:27:08.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.362+0000 7f840cb5e640 1 -- start start 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840cb5e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84080b5460 0x7f84080d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840cb5e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84080d16d0 0x7f8408161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840cb5e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84080d1ae0 con 0x7f84081650b0 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840cb5e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f840806cdc0 con 0x7f84080d16d0 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840cb5e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f840806d0a0 con 0x7f84080b5460 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840659c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f84081650b0 0x7f840806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53078/0 (socket says 172.21.15.94:53078) 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840659c640 1 -- 172.21.15.94:0/2580277688 learned_addr learned my addr 172.21.15.94:0/2580277688 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:08.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f8406d9d640 1 --2- 172.21.15.94:0/2580277688 >> v2:172.21.15.138:3300/0 conn(0x7f84080d16d0 0x7f8408161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f8405d9b640 1 --2- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84080b5460 0x7f84080d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840559a640 1 -- 172.21.15.94:0/2580277688 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 915170414 0 0) 0x7f83f8001490 con 0x7f84081650b0 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840559a640 1 -- 172.21.15.94:0/2580277688 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f0003520 con 0x7f84081650b0 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f8406d9d640 1 -- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84080b5460 msgr2=0x7f84080d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f8406d9d640 1 --2- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84080b5460 0x7f84080d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f8406d9d640 1 -- 172.21.15.94:0/2580277688 >> v1:172.21.15.94:6789/0 conn(0x7f84081650b0 legacy=0x7f840806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f8406d9d640 1 -- 172.21.15.94:0/2580277688 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840806d380 con 0x7f84080d16d0 2024-04-25T16:27:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.364+0000 7f840559a640 1 -- 172.21.15.94:0/2580277688 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2129082837 0 0) 0x7f83f0003520 con 0x7f84081650b0 2024-04-25T16:27:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.365+0000 7f8406d9d640 1 --2- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080d16d0 0x7f8408161fc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84000097a0 tx=0x7f840002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32b47d77048cfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.365+0000 7f840559a640 1 -- 172.21.15.94:0/2580277688 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400037070 con 0x7f84080d16d0 2024-04-25T16:27:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.365+0000 7f840559a640 1 -- 172.21.15.94:0/2580277688 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840002aba0 con 0x7f84080d16d0 2024-04-25T16:27:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.365+0000 7f840559a640 1 -- 172.21.15.94:0/2580277688 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8001bc0 con 0x7f84080d16d0 2024-04-25T16:27:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 -- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080d16d0 msgr2=0x7f8408161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080d16d0 0x7f8408161fc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84000097a0 tx=0x7f840002a110 comp rx=0 tx=0).stop 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 -- 172.21.15.94:0/2580277688 shutdown_connections 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080d16d0 0x7f8408161fc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2580277688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84080b5460 0x7f84080d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 -- 172.21.15.94:0/2580277688 >> 172.21.15.94:0/2580277688 conn(0x7f84080b7e90 msgr2=0x7f84080b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 -- 172.21.15.94:0/2580277688 shutdown_connections 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.366+0000 7f840cb5e640 1 -- 172.21.15.94:0/2580277688 wait complete. 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.368+0000 7f840cb5e640 1 Processor -- start 2024-04-25T16:27:08.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.368+0000 7f840cb5e640 1 -- start start 2024-04-25T16:27:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f840cb5e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080b5460 0x7f84080cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f840cb5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 0x7f84081fb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f840cb5e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84081650b0 0x7f84080ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f840cb5e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f840806de60 con 0x7f84080d16d0 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f840cb5e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f840806d940 con 0x7f84080b5460 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f840cb5e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84080b5870 con 0x7f84081650b0 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f8405d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 0x7f84081fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.369+0000 7f8405d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 0x7f84081fb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47306/0 (socket says 172.21.15.94:47306) 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 -- 172.21.15.94:0/2610096543 learned_addr learned my addr 172.21.15.94:0/2610096543 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f840659c640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080b5460 0x7f84080cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8406d9d640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84081650b0 0x7f84080ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 -- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84081650b0 msgr2=0x7f84080ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84081650b0 0x7f84080ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 -- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080b5460 msgr2=0x7f84080cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080b5460 0x7f84080cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 -- 172.21.15.94:0/2610096543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84080acc20 con 0x7f84080d16d0 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8406d9d640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84081650b0 0x7f84080ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:08.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f8405d9b640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 0x7f84081fb400 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e870 tx=0x7f83fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc017070 con 0x7f84080d16d0 2024-04-25T16:27:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc00c940 con 0x7f84080d16d0 2024-04-25T16:27:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84080acf00 con 0x7f84080d16d0 2024-04-25T16:27:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00cd30 con 0x7f84080d16d0 2024-04-25T16:27:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.370+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84080cc050 con 0x7f84080d16d0 2024-04-25T16:27:08.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.371+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840806a180 con 0x7f84080d16d0 2024-04-25T16:27:08.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.372+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83fc01a070 con 0x7f84080d16d0 2024-04-25T16:27:08.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.372+0000 7f83f6ffd640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f83e80744b0 0x7f83e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.372+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83fc0958c0 con 0x7f84080d16d0 2024-04-25T16:27:08.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.373+0000 7f840659c640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f83e80744b0 0x7f83e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.373+0000 7f840659c640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f83e80744b0 0x7f83e80769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005c00 tx=0x7f83f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.377+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83fc05fc90 con 0x7f84080d16d0 2024-04-25T16:27:08.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.552+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f84080cddb0 con 0x7f84080d16d0 2024-04-25T16:27:08.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.552+0000 7f83f6ffd640 1 -- 172.21.15.94:0/2610096543 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f83fc05f590 con 0x7f84080d16d0 2024-04-25T16:27:08.553 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f83e80744b0 msgr2=0x7f83e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f83e80744b0 0x7f83e80769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005c00 tx=0x7f83f8009040 comp rx=0 tx=0).stop 2024-04-25T16:27:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 msgr2=0x7f84081fb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 0x7f84081fb400 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e870 tx=0x7f83fc00ed30 comp rx=0 tx=0).stop 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 shutdown_connections 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f83e80744b0 0x7f83e80769c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84081650b0 0x7f84080ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84080d16d0 0x7f84081fb400 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 --2- 172.21.15.94:0/2610096543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84080b5460 0x7f84080cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 >> 172.21.15.94:0/2610096543 conn(0x7f84080b7e90 msgr2=0x7f840815ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 shutdown_connections 2024-04-25T16:27:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.554+0000 7f840cb5e640 1 -- 172.21.15.94:0/2610096543 wait complete. 2024-04-25T16:27:08.566 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:08.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.679+0000 7f39f85d5640 1 Processor -- start 2024-04-25T16:27:08.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.679+0000 7f39f85d5640 1 -- start start 2024-04-25T16:27:08.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f85d5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39f00ceb90 0x7f39f00cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f85d5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f00d2270 con 0x7f39f00cf4e0 2024-04-25T16:27:08.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f85d5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f00d1f90 con 0x7f39f00b5420 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f85d5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39f00b34b0 con 0x7f39f00ceb90 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f6b72640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f39f00cf4e0 0x7f39f00b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53084/0 (socket says 172.21.15.94:53084) 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f6b72640 1 -- 172.21.15.94:0/2010819412 learned_addr learned my addr 172.21.15.94:0/2010819412 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f5b70640 1 --2- 172.21.15.94:0/2010819412 >> v2:172.21.15.138:3300/0 conn(0x7f39f00ceb90 0x7f39f00cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239720129 0 0) 0x7f39f00d2270 con 0x7f39f00cf4e0 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39dc003520 con 0x7f39f00cf4e0 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.681+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722873197 0 0) 0x7f39f00d1f90 con 0x7f39f00b5420 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f00d2270 con 0x7f39f00b5420 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f5b70640 1 -- 172.21.15.94:0/2010819412 >> v1:172.21.15.138:6789/0 conn(0x7f39f00b5420 legacy=0x7f39f009c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2193075772 0 0) 0x7f39dc003520 con 0x7f39f00cf4e0 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f5b70640 1 -- 172.21.15.94:0/2010819412 >> v1:172.21.15.94:6789/0 conn(0x7f39f00cf4e0 legacy=0x7f39f00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f5b70640 1 -- 172.21.15.94:0/2010819412 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f009d150 con 0x7f39f00ceb90 2024-04-25T16:27:08.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f5b70640 1 --2- 172.21.15.94:0/2010819412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00ceb90 0x7f39f00cefa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39e00097a0 tx=0x7f39e002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c767204d695a5bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.682+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0037070 con 0x7f39f00ceb90 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e002abc0 con 0x7f39f00ceb90 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f536f640 1 -- 172.21.15.94:0/2010819412 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8001670 con 0x7f39f00ceb90 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 -- 172.21.15.94:0/2010819412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00ceb90 msgr2=0x7f39f00cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2010819412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00ceb90 0x7f39f00cefa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39e00097a0 tx=0x7f39e002a130 comp rx=0 tx=0).stop 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 -- 172.21.15.94:0/2010819412 shutdown_connections 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2010819412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00ceb90 0x7f39f00cefa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 -- 172.21.15.94:0/2010819412 >> 172.21.15.94:0/2010819412 conn(0x7f39f00b5b20 msgr2=0x7f39f00b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 -- 172.21.15.94:0/2010819412 shutdown_connections 2024-04-25T16:27:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.683+0000 7f39f85d5640 1 -- 172.21.15.94:0/2010819412 wait complete. 2024-04-25T16:27:08.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.685+0000 7f39f85d5640 1 Processor -- start 2024-04-25T16:27:08.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.685+0000 7f39f85d5640 1 -- start start 2024-04-25T16:27:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f85d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 0x7f39f00b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f85d5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39f00ceb90 0x7f39f01ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f85d5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00cf4e0 0x7f39f01de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f85d5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00cfed0 con 0x7f39f00b5420 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f85d5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39f009c9c0 con 0x7f39f00cf4e0 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f85d5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00b0c20 con 0x7f39f00ceb90 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.686+0000 7f39f6371640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 0x7f39f00b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 0x7f39f00b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47322/0 (socket says 172.21.15.94:47322) 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 -- 172.21.15.94:0/2890958452 learned_addr learned my addr 172.21.15.94:0/2890958452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6b72640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00cf4e0 0x7f39f01de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 -- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39f00ceb90 msgr2=0x7f39f01ddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f5b70640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39f00ceb90 0x7f39f01ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39f00ceb90 0x7f39f01ddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 -- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00cf4e0 msgr2=0x7f39f01de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00cf4e0 0x7f39f01de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 -- 172.21.15.94:0/2890958452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f00b50b0 con 0x7f39f00b5420 2024-04-25T16:27:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6b72640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00cf4e0 0x7f39f01de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f6371640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 0x7f39f00b4b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c7a0 tx=0x7f39ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec003b70 con 0x7f39f00b5420 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39ec003d10 con 0x7f39f00b5420 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f00c7d00 con 0x7f39f00b5420 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00aa90 con 0x7f39f00b5420 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.687+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f00c82c0 con 0x7f39f00b5420 2024-04-25T16:27:08.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.688+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f006a180 con 0x7f39f00b5420 2024-04-25T16:27:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.689+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f39ec00ac30 con 0x7f39f00b5420 2024-04-25T16:27:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.689+0000 7f39e6ffd640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f39d40744b0 0x7f39d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.689+0000 7f39f5b70640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f39d40744b0 0x7f39d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.689+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f39ec095590 con 0x7f39f00b5420 2024-04-25T16:27:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.690+0000 7f39f5b70640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f39d40744b0 0x7f39d40769c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034000 tx=0x7f39e002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.694+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39ec05f960 con 0x7f39f00b5420 2024-04-25T16:27:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.869+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f39f009ce20 con 0x7f39f00b5420 2024-04-25T16:27:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.869+0000 7f39e6ffd640 1 -- 172.21.15.94:0/2890958452 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f39ec05f260 con 0x7f39f00b5420 2024-04-25T16:27:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f39d40744b0 msgr2=0x7f39d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f39d40744b0 0x7f39d40769c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034000 tx=0x7f39e002e000 comp rx=0 tx=0).stop 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 msgr2=0x7f39f00b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 0x7f39f00b4b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c7a0 tx=0x7f39ec00cc60 comp rx=0 tx=0).stop 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 shutdown_connections 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f39d40744b0 0x7f39d40769c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f00cf4e0 0x7f39f01de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39f00ceb90 0x7f39f01ddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 --2- 172.21.15.94:0/2890958452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39f00b5420 0x7f39f00b4b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 >> 172.21.15.94:0/2890958452 conn(0x7f39f00b5b20 msgr2=0x7f39f00b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 shutdown_connections 2024-04-25T16:27:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.871+0000 7f39f85d5640 1 -- 172.21.15.94:0/2890958452 wait complete. 2024-04-25T16:27:08.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:27:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.989+0000 7f5ce4892640 1 Processor -- start 2024-04-25T16:27:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.989+0000 7f5ce4892640 1 -- start start 2024-04-25T16:27:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5ce4892640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce008d1a0 0x7f5ce008d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5ce4892640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5ce008daf0 0x7f5ce00d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5ce4892640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce00b4530 con 0x7f5ce00b9710 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5ce4892640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce00b42e0 con 0x7f5ce008daf0 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5ce4892640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce00d2430 con 0x7f5ce008d1a0 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cded9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5ce008daf0 0x7f5ce00d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cde59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f5ce00b9710 0x7f5ce00cfe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53090/0 (socket says 172.21.15.94:53090) 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cde59c640 1 -- 172.21.15.94:0/1074622021 learned_addr learned my addr 172.21.15.94:0/1074622021 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cddd9b640 1 --2- 172.21.15.94:0/1074622021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce008d1a0 0x7f5ce008d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cded9d640 1 -- 172.21.15.94:0/1074622021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce008d1a0 msgr2=0x7f5ce008d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cded9d640 1 --2- 172.21.15.94:0/1074622021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce008d1a0 0x7f5ce008d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cded9d640 1 -- 172.21.15.94:0/1074622021 >> v1:172.21.15.94:6789/0 conn(0x7f5ce00b9710 legacy=0x7f5ce00cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.991+0000 7f5cded9d640 1 -- 172.21.15.94:0/1074622021 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce00b4810 con 0x7f5ce008daf0 2024-04-25T16:27:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.992+0000 7f5cded9d640 1 --2- 172.21.15.94:0/1074622021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008daf0 0x7f5ce00d28c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80097a0 tx=0x7f5cd802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77efc578dd5dc33f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.992+0000 7f5cdd59a640 1 -- 172.21.15.94:0/1074622021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8037070 con 0x7f5ce008daf0 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.992+0000 7f5cdd59a640 1 -- 172.21.15.94:0/1074622021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd8032440 con 0x7f5ce008daf0 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.992+0000 7f5cdd59a640 1 -- 172.21.15.94:0/1074622021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8032810 con 0x7f5ce008daf0 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 -- 172.21.15.94:0/1074622021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008daf0 msgr2=0x7f5ce00d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1074622021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008daf0 0x7f5ce00d28c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80097a0 tx=0x7f5cd802a180 comp rx=0 tx=0).stop 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 -- 172.21.15.94:0/1074622021 shutdown_connections 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1074622021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008daf0 0x7f5ce00d28c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1074622021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce008d1a0 0x7f5ce008d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 -- 172.21.15.94:0/1074622021 >> 172.21.15.94:0/1074622021 conn(0x7f5ce00b5a60 msgr2=0x7f5ce00b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 -- 172.21.15.94:0/1074622021 shutdown_connections 2024-04-25T16:27:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.993+0000 7f5ce4892640 1 -- 172.21.15.94:0/1074622021 wait complete. 2024-04-25T16:27:08.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.995+0000 7f5ce4892640 1 Processor -- start 2024-04-25T16:27:08.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.995+0000 7f5ce4892640 1 -- start start 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5ce4892640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 0x7f5ce01d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5ce4892640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce008daf0 0x7f5ce00cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5ce4892640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce00b9710 0x7f5ce0072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5ce4892640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce00b4c70 con 0x7f5ce008d1a0 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5ce4892640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce00b4af0 con 0x7f5ce008daf0 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5ce4892640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce00ce5c0 con 0x7f5ce00b9710 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 0x7f5ce01d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 0x7f5ce01d92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47330/0 (socket says 172.21.15.94:47330) 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 -- 172.21.15.94:0/1845825639 learned_addr learned my addr 172.21.15.94:0/1845825639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cded9d640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce00b9710 0x7f5ce0072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cddd9b640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce008daf0 0x7f5ce00cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:08.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 -- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce00b9710 msgr2=0x7f5ce0072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce00b9710 0x7f5ce0072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 -- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce008daf0 msgr2=0x7f5ce00cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce008daf0 0x7f5ce00cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cde59c640 1 -- 172.21.15.94:0/1845825639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce00cb9b0 con 0x7f5ce008d1a0 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cddd9b640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce008daf0 0x7f5ce00cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.996+0000 7f5cded9d640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce00b9710 0x7f5ce0072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.997+0000 7f5cde59c640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 0x7f5ce01d92b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000cad0 tx=0x7f5cd000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.997+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0017070 con 0x7f5ce008d1a0 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.997+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd000abf0 con 0x7f5ce008d1a0 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.997+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce00cbc90 con 0x7f5ce008d1a0 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.997+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0012400 con 0x7f5ce008d1a0 2024-04-25T16:27:08.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.997+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce00cc250 con 0x7f5ce008d1a0 2024-04-25T16:27:09.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.998+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce006a180 con 0x7f5ce008d1a0 2024-04-25T16:27:09.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.999+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5cd001a070 con 0x7f5ce008d1a0 2024-04-25T16:27:09.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.999+0000 7f5cceffd640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5cc00744b0 0x7f5cc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.999+0000 7f5cddd9b640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5cc00744b0 0x7f5cc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:08.999+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5cd0095250 con 0x7f5ce008d1a0 2024-04-25T16:27:09.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.000+0000 7f5cddd9b640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5cc00744b0 0x7f5cc00769c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4006fd0 tx=0x7f5cd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.004+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5cd005f730 con 0x7f5ce008d1a0 2024-04-25T16:27:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.191+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ce008e440 con 0x7f5ce008d1a0 2024-04-25T16:27:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.191+0000 7f5cceffd640 1 -- 172.21.15.94:0/1845825639 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f5cd005f030 con 0x7f5ce008d1a0 2024-04-25T16:27:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:27:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5cc00744b0 msgr2=0x7f5cc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5cc00744b0 0x7f5cc00769c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4006fd0 tx=0x7f5cd4008040 comp rx=0 tx=0).stop 2024-04-25T16:27:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 msgr2=0x7f5ce01d92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 0x7f5ce01d92b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000cad0 tx=0x7f5cd000cf90 comp rx=0 tx=0).stop 2024-04-25T16:27:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 shutdown_connections 2024-04-25T16:27:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5cc00744b0 0x7f5cc00769c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce00b9710 0x7f5ce0072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce008daf0 0x7f5ce00cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 --2- 172.21.15.94:0/1845825639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce008d1a0 0x7f5ce01d92b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 >> 172.21.15.94:0/1845825639 conn(0x7f5ce00b5a60 msgr2=0x7f5ce00b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 shutdown_connections 2024-04-25T16:27:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.193+0000 7f5ce4892640 1 -- 172.21.15.94:0/1845825639 wait complete. 2024-04-25T16:27:09.205 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:09.205 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:09.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:09.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.366+0000 7ff5ceaaf640 1 Processor -- start 2024-04-25T16:27:09.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.366+0000 7ff5ceaaf640 1 -- start start 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5ceaaf640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff5c80b5230 0x7ff5c8160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5ceaaf640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff5c8164d30 0x7ff5c806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5ceaaf640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80b5a30 con 0x7ff5c8164210 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5ceaaf640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c806ca10 con 0x7ff5c8164d30 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5ceaaf640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c80d5610 con 0x7ff5c80b5230 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5cd04c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff5c8164d30 0x7ff5c806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5c7fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff5c8164210 0x7ff5c8164620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53096/0 (socket says 172.21.15.94:53096) 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5c7fff640 1 -- 172.21.15.94:0/885964509 learned_addr learned my addr 172.21.15.94:0/885964509 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5cc84b640 1 --2- 172.21.15.94:0/885964509 >> v2:172.21.15.138:3300/0 conn(0x7ff5c80b5230 0x7ff5c8160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5cd04c640 1 -- 172.21.15.94:0/885964509 >> v2:172.21.15.138:3300/0 conn(0x7ff5c80b5230 msgr2=0x7ff5c8160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5cd04c640 1 --2- 172.21.15.94:0/885964509 >> v2:172.21.15.138:3300/0 conn(0x7ff5c80b5230 0x7ff5c8160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5cd04c640 1 -- 172.21.15.94:0/885964509 >> v1:172.21.15.94:6789/0 conn(0x7ff5c8164210 legacy=0x7ff5c8164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5cd04c640 1 -- 172.21.15.94:0/885964509 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80d58f0 con 0x7ff5c8164d30 2024-04-25T16:27:09.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.368+0000 7ff5c77fe640 1 -- 172.21.15.94:0/885964509 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494628981 0 0) 0x7ff5bc001110 con 0x7ff5c8164210 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.369+0000 7ff5cd04c640 1 --2- 172.21.15.94:0/885964509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c8164d30 0x7ff5c806cea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5c808f150 tx=0x7ff5c002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe44963698d8067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.369+0000 7ff5c77fe640 1 -- 172.21.15.94:0/885964509 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0007740 con 0x7ff5c8164d30 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.369+0000 7ff5c77fe640 1 -- 172.21.15.94:0/885964509 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c00078e0 con 0x7ff5c8164d30 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.369+0000 7ff5c77fe640 1 -- 172.21.15.94:0/885964509 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0007cb0 con 0x7ff5c8164d30 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/885964509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c8164d30 msgr2=0x7ff5c806cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/885964509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c8164d30 0x7ff5c806cea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5c808f150 tx=0x7ff5c002a5f0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/885964509 shutdown_connections 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/885964509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c8164d30 0x7ff5c806cea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/885964509 >> v2:172.21.15.138:3300/0 conn(0x7ff5c80b5230 0x7ff5c8160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/885964509 >> 172.21.15.94:0/885964509 conn(0x7ff5c80d3570 msgr2=0x7ff5c80b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/885964509 shutdown_connections 2024-04-25T16:27:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.370+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/885964509 wait complete. 2024-04-25T16:27:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.371+0000 7ff5ceaaf640 1 Processor -- start 2024-04-25T16:27:09.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.371+0000 7ff5ceaaf640 1 -- start start 2024-04-25T16:27:09.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5ceaaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 0x7ff5c80cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5ceaaf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5c8164210 0x7ff5c815f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5ceaaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5c8164d30 0x7ff5c8161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5ceaaf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80b9870 con 0x7ff5c80b5230 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5ceaaf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80b96f0 con 0x7ff5c8164d30 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5ceaaf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c806d3e0 con 0x7ff5c8164210 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 0x7ff5c80cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 0x7ff5c80cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47354/0 (socket says 172.21.15.94:47354) 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 -- 172.21.15.94:0/3790520753 learned_addr learned my addr 172.21.15.94:0/3790520753 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cd04c640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5c8164d30 0x7ff5c8161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5c7fff640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5c8164210 0x7ff5c815f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 -- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5c8164210 msgr2=0x7ff5c815f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5c8164210 0x7ff5c815f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 -- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5c8164d30 msgr2=0x7ff5c8161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5c8164d30 0x7ff5c8161d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 -- 172.21.15.94:0/3790520753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80d01d0 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5c7fff640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5c8164210 0x7ff5c815f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cd04c640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5c8164d30 0x7ff5c8161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.373+0000 7ff5cc84b640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 0x7ff5c80cf830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800c7f0 tx=0x7ff5b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.374+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8015070 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.374+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8004220 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.374+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c80c9ed0 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.374+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007620 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.374+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c80ca490 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.375+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c80d05a0 con 0x7ff5c80b5230 2024-04-25T16:27:09.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.375+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff5b80077c0 con 0x7ff5c80b5230 2024-04-25T16:27:09.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.376+0000 7ff5c57fa640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff5b40744b0 0x7ff5b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.376+0000 7ff5c7fff640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff5b40744b0 0x7ff5b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.376+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff5b8095320 con 0x7ff5c80b5230 2024-04-25T16:27:09.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.376+0000 7ff5c7fff640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff5b40744b0 0x7ff5b40769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5c806ffe0 tx=0x7ff5bc007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.380+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5b805f800 con 0x7ff5c80b5230 2024-04-25T16:27:09.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.553+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5c806a180 con 0x7ff5c80b5230 2024-04-25T16:27:09.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.555+0000 7ff5c57fa640 1 -- 172.21.15.94:0/3790520753 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7ff5b805f100 con 0x7ff5c80b5230 2024-04-25T16:27:09.556 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:09.556 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff5b40744b0 msgr2=0x7ff5b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff5b40744b0 0x7ff5b40769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5c806ffe0 tx=0x7ff5bc007cc0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 msgr2=0x7ff5c80cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 0x7ff5c80cf830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800c7f0 tx=0x7ff5b800ccb0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 shutdown_connections 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff5b40744b0 0x7ff5b40769c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5c8164d30 0x7ff5c8161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5c8164210 0x7ff5c815f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 --2- 172.21.15.94:0/3790520753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b5230 0x7ff5c80cf830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 >> 172.21.15.94:0/3790520753 conn(0x7ff5c80d3570 msgr2=0x7ff5c80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 shutdown_connections 2024-04-25T16:27:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.557+0000 7ff5ceaaf640 1 -- 172.21.15.94:0/3790520753 wait complete. 2024-04-25T16:27:09.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:09.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:09.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.680+0000 7fb23aebd640 1 Processor -- start 2024-04-25T16:27:09.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.680+0000 7fb23aebd640 1 -- start start 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb23aebd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb2340b5420 0x7fb23409c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb23aebd640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2340d2270 con 0x7fb234163450 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb23aebd640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2340d1f90 con 0x7fb23409c7f0 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb23aebd640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb2340b34b0 con 0x7fb2340b5420 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb238c59640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb2340b5420 0x7fb23409c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb23945a640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb234163450 0x7fb2340b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53112/0 (socket says 172.21.15.94:53112) 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb23945a640 1 -- 172.21.15.94:0/2629037930 learned_addr learned my addr 172.21.15.94:0/2629037930 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb238c59640 1 -- 172.21.15.94:0/2629037930 >> v1:172.21.15.138:6789/0 conn(0x7fb23409c7f0 legacy=0x7fb234162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb238c59640 1 -- 172.21.15.94:0/2629037930 >> v1:172.21.15.94:6789/0 conn(0x7fb234163450 legacy=0x7fb2340b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb238c59640 1 -- 172.21.15.94:0/2629037930 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23409d150 con 0x7fb2340b5420 2024-04-25T16:27:09.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.682+0000 7fb2337fe640 1 -- 172.21.15.94:0/2629037930 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 74857771 0 0) 0x7fb2340d2270 con 0x7fb234163450 2024-04-25T16:27:09.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.683+0000 7fb238c59640 1 --2- 172.21.15.94:0/2629037930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2340b5420 0x7fb23409c2b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2240097a0 tx=0x7fb22402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84149f52bbbd240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.683+0000 7fb2337fe640 1 -- 172.21.15.94:0/2629037930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224037070 con 0x7fb2340b5420 2024-04-25T16:27:09.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.683+0000 7fb2337fe640 1 -- 172.21.15.94:0/2629037930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22402ae50 con 0x7fb2340b5420 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.683+0000 7fb2337fe640 1 -- 172.21.15.94:0/2629037930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224032660 con 0x7fb2340b5420 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 -- 172.21.15.94:0/2629037930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2340b5420 msgr2=0x7fb23409c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 --2- 172.21.15.94:0/2629037930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2340b5420 0x7fb23409c2b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2240097a0 tx=0x7fb22402a140 comp rx=0 tx=0).stop 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 -- 172.21.15.94:0/2629037930 shutdown_connections 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 --2- 172.21.15.94:0/2629037930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2340b5420 0x7fb23409c2b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 -- 172.21.15.94:0/2629037930 >> 172.21.15.94:0/2629037930 conn(0x7fb2340b5b20 msgr2=0x7fb2340b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 -- 172.21.15.94:0/2629037930 shutdown_connections 2024-04-25T16:27:09.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.684+0000 7fb23aebd640 1 -- 172.21.15.94:0/2629037930 wait complete. 2024-04-25T16:27:09.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.685+0000 7fb23aebd640 1 Processor -- start 2024-04-25T16:27:09.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.685+0000 7fb23aebd640 1 -- start start 2024-04-25T16:27:09.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23aebd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 0x7fb2340cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23aebd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2340b5420 0x7fb23414df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23aebd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb234163450 0x7fb2340ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23aebd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b0c20 con 0x7fb23409c7f0 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23aebd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340cfed0 con 0x7fb234163450 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23aebd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2340b0da0 con 0x7fb2340b5420 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 0x7fb2340cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 0x7fb2340cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47364/0 (socket says 172.21.15.94:47364) 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 -- 172.21.15.94:0/3397617002 learned_addr learned my addr 172.21.15.94:0/3397617002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb233fff640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2340b5420 0x7fb23414df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb23945a640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb234163450 0x7fb2340ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 -- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2340b5420 msgr2=0x7fb23414df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2340b5420 0x7fb23414df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 -- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb234163450 msgr2=0x7fb2340ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb234163450 0x7fb2340ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 -- 172.21.15.94:0/3397617002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2340cd420 con 0x7fb23409c7f0 2024-04-25T16:27:09.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.687+0000 7fb238c59640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 0x7fb2340cb200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb22402a730 tx=0x7fb2240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.688+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224042070 con 0x7fb23409c7f0 2024-04-25T16:27:09.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.688+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22403e670 con 0x7fb23409c7f0 2024-04-25T16:27:09.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.688+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2340cd700 con 0x7fb23409c7f0 2024-04-25T16:27:09.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.688+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224037040 con 0x7fb23409c7f0 2024-04-25T16:27:09.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.688+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2340cdcc0 con 0x7fb23409c7f0 2024-04-25T16:27:09.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.689+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23406a180 con 0x7fb23409c7f0 2024-04-25T16:27:09.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.689+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb224045070 con 0x7fb23409c7f0 2024-04-25T16:27:09.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.690+0000 7fb2317fa640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb2200744b0 0x7fb2200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.690+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb2240b5cf0 con 0x7fb23409c7f0 2024-04-25T16:27:09.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.690+0000 7fb233fff640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb2200744b0 0x7fb2200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.690+0000 7fb233fff640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb2200744b0 0x7fb2200769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb228006fd0 tx=0x7fb228008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.694+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2240ba070 con 0x7fb23409c7f0 2024-04-25T16:27:09.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.868+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb23409ce20 con 0x7fb23409c7f0 2024-04-25T16:27:09.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.869+0000 7fb2317fa640 1 -- 172.21.15.94:0/3397617002 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fb2240801d0 con 0x7fb23409c7f0 2024-04-25T16:27:09.870 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb2200744b0 msgr2=0x7fb2200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb2200744b0 0x7fb2200769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb228006fd0 tx=0x7fb228008040 comp rx=0 tx=0).stop 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 msgr2=0x7fb2340cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 0x7fb2340cb200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb22402a730 tx=0x7fb2240046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 shutdown_connections 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb2200744b0 0x7fb2200769c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb234163450 0x7fb2340ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2340b5420 0x7fb23414df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 --2- 172.21.15.94:0/3397617002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23409c7f0 0x7fb2340cb200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 >> 172.21.15.94:0/3397617002 conn(0x7fb2340b5b20 msgr2=0x7fb2340724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:09.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 shutdown_connections 2024-04-25T16:27:09.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.871+0000 7fb23aebd640 1 -- 172.21.15.94:0/3397617002 wait complete. 2024-04-25T16:27:09.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:09.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.992+0000 7faee571c640 1 Processor -- start 2024-04-25T16:27:09.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.992+0000 7faee571c640 1 -- start start 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faee571c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faee00d4aa0 0x7faee00b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faee571c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faee006d960 0x7faee00b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faee571c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00b9cc0 0x7faee00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faee571c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faee006c8c0 con 0x7faee006d960 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faee571c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faee006cba0 con 0x7faee00d4aa0 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faee571c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faee006ce80 con 0x7faee00b9cc0 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.993+0000 7faede7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faee006d960 0x7faee00b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faee006d960 0x7faee00b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47376/0 (socket says 172.21.15.94:47376) 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 -- 172.21.15.94:0/3851969535 learned_addr learned my addr 172.21.15.94:0/3851969535 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faedeffd640 1 --2- 172.21.15.94:0/3851969535 >> v2:172.21.15.138:3300/0 conn(0x7faee00d4aa0 0x7faee00b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faedf7fe640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00b9cc0 0x7faee00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 -- 172.21.15.94:0/3851969535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00b9cc0 msgr2=0x7faee00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00b9cc0 0x7faee00d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 -- 172.21.15.94:0/3851969535 >> v2:172.21.15.138:3300/0 conn(0x7faee00d4aa0 msgr2=0x7faee00b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 --2- 172.21.15.94:0/3851969535 >> v2:172.21.15.138:3300/0 conn(0x7faee00d4aa0 0x7faee00b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faede7fc640 1 -- 172.21.15.94:0/3851969535 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee006d160 con 0x7faee006d960 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.994+0000 7faedf7fe640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00b9cc0 0x7faee00d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faede7fc640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee006d960 0x7faee00b9780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faee006e0b0 tx=0x7faed402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589e5dbea7c137b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faeddffb640 1 -- 172.21.15.94:0/3851969535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037070 con 0x7faee006d960 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faeddffb640 1 -- 172.21.15.94:0/3851969535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed402ae00 con 0x7faee006d960 2024-04-25T16:27:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faeddffb640 1 -- 172.21.15.94:0/3851969535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4032660 con 0x7faee006d960 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 -- 172.21.15.94:0/3851969535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee006d960 msgr2=0x7faee00b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee006d960 0x7faee00b9780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faee006e0b0 tx=0x7faed402a0b0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 -- 172.21.15.94:0/3851969535 shutdown_connections 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00b9cc0 0x7faee00d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 --2- 172.21.15.94:0/3851969535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee006d960 0x7faee00b9780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 --2- 172.21.15.94:0/3851969535 >> v2:172.21.15.138:3300/0 conn(0x7faee00d4aa0 0x7faee00b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 -- 172.21.15.94:0/3851969535 >> 172.21.15.94:0/3851969535 conn(0x7faee00b7e60 msgr2=0x7faee00b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 -- 172.21.15.94:0/3851969535 shutdown_connections 2024-04-25T16:27:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.995+0000 7faee571c640 1 -- 172.21.15.94:0/3851969535 wait complete. 2024-04-25T16:27:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.997+0000 7faee571c640 1 Processor -- start 2024-04-25T16:27:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.997+0000 7faee571c640 1 -- start start 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee006d960 0x7faee01f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 0x7faee01f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00d4aa0 0x7faee00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faee006dd70 con 0x7faee00b9cc0 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faee006c570 con 0x7faee006d960 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faee006def0 con 0x7faee00d4aa0 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 0x7faee01f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 0x7faee01f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47390/0 (socket says 172.21.15.94:47390) 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 -- 172.21.15.94:0/1099218397 learned_addr learned my addr 172.21.15.94:0/1099218397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faedf7fe640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00d4aa0 0x7faee00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faedeffd640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee006d960 0x7faee01f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 -- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00d4aa0 msgr2=0x7faee00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00d4aa0 0x7faee00cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 -- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee006d960 msgr2=0x7faee01f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee006d960 0x7faee01f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 -- 172.21.15.94:0/1099218397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee01fa950 con 0x7faee00b9cc0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faede7fc640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 0x7faee01f9e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faed4000c00 tx=0x7faed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4041030 con 0x7faee00b9cc0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed402ac00 con 0x7faee00b9cc0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee01fac30 con 0x7faee00b9cc0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037040 con 0x7faee00b9cc0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:09.999+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee00ad0b0 con 0x7faee00b9cc0 2024-04-25T16:27:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.000+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee00acbc0 con 0x7faee00b9cc0 2024-04-25T16:27:10.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.001+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faed4044070 con 0x7faee00b9cc0 2024-04-25T16:27:10.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.001+0000 7faec77fe640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faecc0744e0 0x7faecc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.001+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faed40b56c0 con 0x7faee00b9cc0 2024-04-25T16:27:10.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.001+0000 7faedeffd640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faecc0744e0 0x7faecc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.002+0000 7faedeffd640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faecc0744e0 0x7faecc0769f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faed0006fd0 tx=0x7faed0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.006+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faed404b070 con 0x7faee00b9cc0 2024-04-25T16:27:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.180+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faee006a180 con 0x7faee00b9cc0 2024-04-25T16:27:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.181+0000 7faec77fe640 1 -- 172.21.15.94:0/1099218397 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7faed407fa90 con 0x7faee00b9cc0 2024-04-25T16:27:10.181 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faecc0744e0 msgr2=0x7faecc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faecc0744e0 0x7faecc0769f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faed0006fd0 tx=0x7faed0008040 comp rx=0 tx=0).stop 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 msgr2=0x7faee01f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 0x7faee01f9e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faed4000c00 tx=0x7faed40046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 shutdown_connections 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faecc0744e0 0x7faecc0769f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faee00d4aa0 0x7faee00cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee00b9cc0 0x7faee01f9e90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 --2- 172.21.15.94:0/1099218397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee006d960 0x7faee01f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 >> 172.21.15.94:0/1099218397 conn(0x7faee00b7e60 msgr2=0x7faee0160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 shutdown_connections 2024-04-25T16:27:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.183+0000 7faee571c640 1 -- 172.21.15.94:0/1099218397 wait complete. 2024-04-25T16:27:10.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.304+0000 7f26ed94e640 1 Processor -- start 2024-04-25T16:27:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.304+0000 7f26ed94e640 1 -- start start 2024-04-25T16:27:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.305+0000 7f26ed94e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26e80b5710 0x7f26e81608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.305+0000 7f26ed94e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f26e80d4a20 0x7f26e80d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.305+0000 7f26ed94e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e8161930 con 0x7f26e80b4c70 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.305+0000 7f26ed94e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f26e8161c10 con 0x7f26e80d4a20 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.305+0000 7f26ed94e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f26e8164d10 con 0x7f26e80b5710 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.305+0000 7f26e77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f26e80d4a20 0x7f26e80d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26dffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f26e80b4c70 0x7f26e8160e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53116/0 (socket says 172.21.15.94:53116) 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26dffff640 1 -- 172.21.15.94:0/692770087 learned_addr learned my addr 172.21.15.94:0/692770087 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e6ffd640 1 --2- 172.21.15.94:0/692770087 >> v2:172.21.15.138:3300/0 conn(0x7f26e80b5710 0x7f26e81608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e77fe640 1 -- 172.21.15.94:0/692770087 >> v2:172.21.15.138:3300/0 conn(0x7f26e80b5710 msgr2=0x7f26e81608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e77fe640 1 --2- 172.21.15.94:0/692770087 >> v2:172.21.15.138:3300/0 conn(0x7f26e80b5710 0x7f26e81608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e77fe640 1 -- 172.21.15.94:0/692770087 >> v1:172.21.15.94:6789/0 conn(0x7f26e80b4c70 legacy=0x7f26e8160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e77fe640 1 -- 172.21.15.94:0/692770087 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8164ff0 con 0x7f26e80d4a20 2024-04-25T16:27:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e6ffd640 1 --2- 172.21.15.94:0/692770087 >> v2:172.21.15.138:3300/0 conn(0x7f26e80b5710 0x7f26e81608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.306+0000 7f26e77fe640 1 --2- 172.21.15.94:0/692770087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80d4a20 0x7f26e80d4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f26e808ec70 tx=0x7f26e002a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f8d02276713e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26e67fc640 1 -- 172.21.15.94:0/692770087 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0037070 con 0x7f26e80d4a20 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26e67fc640 1 -- 172.21.15.94:0/692770087 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0032640 con 0x7f26e80d4a20 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26e67fc640 1 -- 172.21.15.94:0/692770087 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0032a10 con 0x7f26e80d4a20 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 -- 172.21.15.94:0/692770087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80d4a20 msgr2=0x7f26e80d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 --2- 172.21.15.94:0/692770087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80d4a20 0x7f26e80d4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f26e808ec70 tx=0x7f26e002a460 comp rx=0 tx=0).stop 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 -- 172.21.15.94:0/692770087 shutdown_connections 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 --2- 172.21.15.94:0/692770087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80d4a20 0x7f26e80d4e30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 --2- 172.21.15.94:0/692770087 >> v2:172.21.15.138:3300/0 conn(0x7f26e80b5710 0x7f26e81608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 -- 172.21.15.94:0/692770087 >> 172.21.15.94:0/692770087 conn(0x7f26e80d6930 msgr2=0x7f26e80b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.307+0000 7f26ed94e640 1 -- 172.21.15.94:0/692770087 shutdown_connections 2024-04-25T16:27:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.308+0000 7f26ed94e640 1 -- 172.21.15.94:0/692770087 wait complete. 2024-04-25T16:27:10.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.309+0000 7f26ed94e640 1 Processor -- start 2024-04-25T16:27:10.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.309+0000 7f26ed94e640 1 -- start start 2024-04-25T16:27:10.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.310+0000 7f26ed94e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 0x7f26e815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.310+0000 7f26ed94e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26e80b5710 0x7f26e815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.310+0000 7f26ed94e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26e80d4a20 0x7f26e80ba790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.310+0000 7f26ed94e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8161530 con 0x7f26e80b4c70 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26ed94e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80b51c0 con 0x7f26e80b5710 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26ed94e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f26e81616b0 con 0x7f26e80d4a20 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 0x7f26e815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 0x7f26e815fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47398/0 (socket says 172.21.15.94:47398) 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 -- 172.21.15.94:0/2611859140 learned_addr learned my addr 172.21.15.94:0/2611859140 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26dffff640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26e80b5710 0x7f26e815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e77fe640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26e80d4a20 0x7f26e80ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 -- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26e80d4a20 msgr2=0x7f26e80ba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26e80d4a20 0x7f26e80ba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 -- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26e80b5710 msgr2=0x7f26e815df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26e80b5710 0x7f26e815df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 -- 172.21.15.94:0/2611859140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e80cb900 con 0x7f26e80b4c70 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26dffff640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26e80b5710 0x7f26e815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e6ffd640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 0x7f26e815fa10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f26d800d800 tx=0x7f26d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d80096a0 con 0x7f26e80b4c70 2024-04-25T16:27:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d8019070 con 0x7f26e80b4c70 2024-04-25T16:27:10.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e815e4a0 con 0x7f26e80b4c70 2024-04-25T16:27:10.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8009a80 con 0x7f26e80b4c70 2024-04-25T16:27:10.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.311+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e815ec90 con 0x7f26e80b4c70 2024-04-25T16:27:10.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.312+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e806a180 con 0x7f26e80b4c70 2024-04-25T16:27:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.313+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f26d8009c20 con 0x7f26e80b4c70 2024-04-25T16:27:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.314+0000 7f26e4ff9640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f26c80744b0 0x7f26c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.314+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f26d80959a0 con 0x7f26e80b4c70 2024-04-25T16:27:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.314+0000 7f26dffff640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f26c80744b0 0x7f26c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.314+0000 7f26dffff640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f26c80744b0 0x7f26c80769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26d0007fd0 tx=0x7f26d0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.318+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26d805fdf0 con 0x7f26e80b4c70 2024-04-25T16:27:10.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.494+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26e80d45c0 con 0x7f26e80b4c70 2024-04-25T16:27:10.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.494+0000 7f26e4ff9640 1 -- 172.21.15.94:0/2611859140 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f26d8066e20 con 0x7f26e80b4c70 2024-04-25T16:27:10.495 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f26c80744b0 msgr2=0x7f26c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f26c80744b0 0x7f26c80769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26d0007fd0 tx=0x7f26d0007d40 comp rx=0 tx=0).stop 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 msgr2=0x7f26e815fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 0x7f26e815fa10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f26d800d800 tx=0x7f26d800dcc0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 shutdown_connections 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f26c80744b0 0x7f26c80769c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26e80d4a20 0x7f26e80ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26e80b5710 0x7f26e815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 --2- 172.21.15.94:0/2611859140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e80b4c70 0x7f26e815fa10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 >> 172.21.15.94:0/2611859140 conn(0x7f26e80d6930 msgr2=0x7f26e815b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 shutdown_connections 2024-04-25T16:27:10.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.496+0000 7f26ed94e640 1 -- 172.21.15.94:0/2611859140 wait complete. 2024-04-25T16:27:10.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:10.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.618+0000 7fd3be0b6640 1 Processor -- start 2024-04-25T16:27:10.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.618+0000 7fd3be0b6640 1 -- start start 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3be0b6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd3b806f130 0x7fd3b809c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3be0b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3be0b6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3b80cf7d0 0x7fd3b80cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3be0b6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b806f540 con 0x7fd3b806f130 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3be0b6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b806e860 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3be0b6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b806f7c0 con 0x7fd3b806df10 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3b77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd3b806f130 0x7fd3b809c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3b77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd3b806f130 0x7fd3b809c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47412/0 (socket says 172.21.15.94:47412) 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3b77fe640 1 -- 172.21.15.94:0/3490694903 learned_addr learned my addr 172.21.15.94:0/3490694903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3b6ffd640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.619+0000 7fd3b7fff640 1 --2- 172.21.15.94:0/3490694903 >> v2:172.21.15.138:3300/0 conn(0x7fd3b80cf7d0 0x7fd3b80cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b77fe640 1 -- 172.21.15.94:0/3490694903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 msgr2=0x7fd3b806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b77fe640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b77fe640 1 -- 172.21.15.94:0/3490694903 >> v2:172.21.15.138:3300/0 conn(0x7fd3b80cf7d0 msgr2=0x7fd3b80cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b77fe640 1 --2- 172.21.15.94:0/3490694903 >> v2:172.21.15.138:3300/0 conn(0x7fd3b80cf7d0 0x7fd3b80cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b77fe640 1 -- 172.21.15.94:0/3490694903 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b8070670 con 0x7fd3b806f130 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b7fff640 1 --2- 172.21.15.94:0/3490694903 >> v2:172.21.15.138:3300/0 conn(0x7fd3b80cf7d0 0x7fd3b80cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b6ffd640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b77fe640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b806f130 0x7fd3b809c530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8073440 tx=0x7fd3a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d914131c05b8990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.620+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3490694903 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8037070 con 0x7fd3b806f130 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3490694903 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a8032440 con 0x7fd3b806f130 2024-04-25T16:27:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3490694903 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8032810 con 0x7fd3b806f130 2024-04-25T16:27:10.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3490694903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b806f130 msgr2=0x7fd3b809c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b806f130 0x7fd3b809c530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8073440 tx=0x7fd3a802a1e0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3490694903 shutdown_connections 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3490694903 >> v2:172.21.15.138:3300/0 conn(0x7fd3b80cf7d0 0x7fd3b80cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3490694903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b806f130 0x7fd3b809c530 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3490694903 >> 172.21.15.94:0/3490694903 conn(0x7fd3b80b7b60 msgr2=0x7fd3b80b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3490694903 shutdown_connections 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.621+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3490694903 wait complete. 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.623+0000 7fd3be0b6640 1 Processor -- start 2024-04-25T16:27:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.623+0000 7fd3be0b6640 1 -- start start 2024-04-25T16:27:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.624+0000 7fd3be0b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b80d3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.624+0000 7fd3be0b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b806f130 0x7fd3b8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.624+0000 7fd3be0b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 0x7fd3b80ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.624+0000 7fd3be0b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b815d7e0 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.624+0000 7fd3be0b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b80cdf50 con 0x7fd3b806f130 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3be0b6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b815f850 con 0x7fd3b806df10 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 0x7fd3b80ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 0x7fd3b80ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47414/0 (socket says 172.21.15.94:47414) 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3910637913 learned_addr learned my addr 172.21.15.94:0/3910637913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b6ffd640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b806f130 0x7fd3b8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b77fe640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b80d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 msgr2=0x7fd3b80d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b80d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b806f130 msgr2=0x7fd3b8157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b806f130 0x7fd3b8157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3910637913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80cc900 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3b7fff640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 0x7fd3b80ca5c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007df0 tx=0x7fd3b000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b001a070 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0003990 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b80ccbe0 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0003d80 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.625+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b815f590 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.627+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b806a180 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.627+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3b001d070 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.627+0000 7fd39ffff640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd3a40744b0 0x7fd3a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.628+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3b00955c0 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.628+0000 7fd3b77fe640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd3a40744b0 0x7fd3a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.628+0000 7fd3b77fe640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd3a40744b0 0x7fd3a40769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8034000 tx=0x7fd3a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.632+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3b0010070 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.807+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd3b8159c80 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.807+0000 7fd39ffff640 1 -- 172.21.15.94:0/3910637913 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fd3b005f4e0 con 0x7fd3b80cf7d0 2024-04-25T16:27:10.808 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd3a40744b0 msgr2=0x7fd3a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd3a40744b0 0x7fd3a40769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8034000 tx=0x7fd3a802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 msgr2=0x7fd3b80ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 0x7fd3b80ca5c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007df0 tx=0x7fd3b000d4a0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 shutdown_connections 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd3a40744b0 0x7fd3a40769c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b80cf7d0 0x7fd3b80ca5c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b806f130 0x7fd3b8157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 --2- 172.21.15.94:0/3910637913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3b806df10 0x7fd3b80d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 >> 172.21.15.94:0/3910637913 conn(0x7fd3b80b7b60 msgr2=0x7fd3b80cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:10.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 shutdown_connections 2024-04-25T16:27:10.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.809+0000 7fd3be0b6640 1 -- 172.21.15.94:0/3910637913 wait complete. 2024-04-25T16:27:10.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:27:10.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.930+0000 7fa86e53f640 1 Processor -- start 2024-04-25T16:27:10.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.930+0000 7fa86e53f640 1 -- start start 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86e53f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa86809c5a0 0x7fa8680b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86e53f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa8680ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86e53f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa86815ec10 con 0x7fa8680ceef0 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86e53f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa8680b50b0 con 0x7fa86809c5a0 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86e53f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680b5350 con 0x7fa86815e800 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa867fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa86809c5a0 0x7fa8680b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86cadc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa8680ceef0 0x7fa8680cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:53118/0 (socket says 172.21.15.94:53118) 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa86cadc640 1 -- 172.21.15.94:0/1673513908 learned_addr learned my addr 172.21.15.94:0/1673513908 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa867fff640 1 -- 172.21.15.94:0/1673513908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 msgr2=0x7fa8680ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa8677fe640 1 --2- 172.21.15.94:0/1673513908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa8680ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa867fff640 1 --2- 172.21.15.94:0/1673513908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa8680ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa867fff640 1 -- 172.21.15.94:0/1673513908 >> v1:172.21.15.94:6789/0 conn(0x7fa8680ceef0 legacy=0x7fa8680cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.931+0000 7fa866ffd640 1 -- 172.21.15.94:0/1673513908 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2431133129 0 0) 0x7fa86815ec10 con 0x7fa8680ceef0 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.932+0000 7fa867fff640 1 -- 172.21.15.94:0/1673513908 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86806cff0 con 0x7fa86809c5a0 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.932+0000 7fa867fff640 1 --2- 172.21.15.94:0/1673513908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86809c5a0 0x7fa8680b4a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8580097a0 tx=0x7fa85802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a4c42dbc054cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.932+0000 7fa866ffd640 1 -- 172.21.15.94:0/1673513908 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858037070 con 0x7fa86809c5a0 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.932+0000 7fa866ffd640 1 -- 172.21.15.94:0/1673513908 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858032440 con 0x7fa86809c5a0 2024-04-25T16:27:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.932+0000 7fa866ffd640 1 -- 172.21.15.94:0/1673513908 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858032810 con 0x7fa86809c5a0 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 -- 172.21.15.94:0/1673513908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86809c5a0 msgr2=0x7fa8680b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 --2- 172.21.15.94:0/1673513908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86809c5a0 0x7fa8680b4a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8580097a0 tx=0x7fa85802a210 comp rx=0 tx=0).stop 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 -- 172.21.15.94:0/1673513908 shutdown_connections 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 --2- 172.21.15.94:0/1673513908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa8680ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 --2- 172.21.15.94:0/1673513908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa86809c5a0 0x7fa8680b4a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 -- 172.21.15.94:0/1673513908 >> 172.21.15.94:0/1673513908 conn(0x7fa8680b7c70 msgr2=0x7fa8680b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 -- 172.21.15.94:0/1673513908 shutdown_connections 2024-04-25T16:27:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.933+0000 7fa86e53f640 1 -- 172.21.15.94:0/1673513908 wait complete. 2024-04-25T16:27:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.935+0000 7fa86e53f640 1 Processor -- start 2024-04-25T16:27:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.935+0000 7fa86e53f640 1 -- start start 2024-04-25T16:27:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa86e53f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa86809c5a0 0x7fa868161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa86e53f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 0x7fa8680c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa86e53f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa868160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa86e53f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa86806e800 con 0x7fa8680ceef0 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa86e53f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680cf300 con 0x7fa86809c5a0 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa86e53f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa86806e980 con 0x7fa86815e800 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.936+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 0x7fa8680c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 0x7fa8680c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47428/0 (socket says 172.21.15.94:47428) 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 -- 172.21.15.94:0/4024776460 learned_addr learned my addr 172.21.15.94:0/4024776460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa867fff640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa86809c5a0 0x7fa868161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa86cadc640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa868160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 -- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 msgr2=0x7fa868160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa868160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 -- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa86809c5a0 msgr2=0x7fa868161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa86809c5a0 0x7fa868161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 -- 172.21.15.94:0/4024776460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86815fdb0 con 0x7fa8680ceef0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa867fff640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa86809c5a0 0x7fa868161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa86cadc640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa868160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa8677fe640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 0x7fa8680c5980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c820 tx=0x7fa85c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c015070 con 0x7fa8680ceef0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85c004020 con 0x7fa8680ceef0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa868160090 con 0x7fa8680ceef0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0043f0 con 0x7fa8680ceef0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.937+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa868160650 con 0x7fa8680ceef0 2024-04-25T16:27:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.938+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86806a180 con 0x7fa8680ceef0 2024-04-25T16:27:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.939+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa85c007830 con 0x7fa8680ceef0 2024-04-25T16:27:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.939+0000 7fa864ff9640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8540744e0 0x7fa8540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.939+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa85c0953a0 con 0x7fa8680ceef0 2024-04-25T16:27:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.939+0000 7fa867fff640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8540744e0 0x7fa8540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.940+0000 7fa867fff640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8540744e0 0x7fa8540769f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa858034000 tx=0x7fa85802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:10.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:10.944+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa85c05f750 con 0x7fa8680ceef0 2024-04-25T16:27:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.133+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa86815fb40 con 0x7fa8680ceef0 2024-04-25T16:27:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.134+0000 7fa864ff9640 1 -- 172.21.15.94:0/4024776460 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fa85c05f050 con 0x7fa8680ceef0 2024-04-25T16:27:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:27:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8540744e0 msgr2=0x7fa8540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8540744e0 0x7fa8540769f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa858034000 tx=0x7fa85802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 msgr2=0x7fa8680c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 0x7fa8680c5980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c820 tx=0x7fa85c00cce0 comp rx=0 tx=0).stop 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 shutdown_connections 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8540744e0 0x7fa8540769f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa86815e800 0x7fa868160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8680ceef0 0x7fa8680c5980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 --2- 172.21.15.94:0/4024776460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa86809c5a0 0x7fa868161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 >> 172.21.15.94:0/4024776460 conn(0x7fa8680b7c70 msgr2=0x7fa86815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 shutdown_connections 2024-04-25T16:27:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:11.136+0000 7fa86e53f640 1 -- 172.21.15.94:0/4024776460 wait complete. 2024-04-25T16:27:11.147 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-04-25T16:27:11.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:11.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:27:11.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[41657]: fuse finished with error 0 and tester_r 0 2024-04-25T16:27:11.171 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:27:17.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:27:17.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:17.273 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T16:27:17.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:17.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.456+0000 7ffa5a14d640 1 Processor -- start 2024-04-25T16:27:17.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.456+0000 7ffa5a14d640 1 -- start start 2024-04-25T16:27:17.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa5a14d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffa540b0860 0x7ffa54160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa5a14d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5406f5f0 con 0x7ffa5406d580 2024-04-25T16:27:17.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa5a14d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5406f8d0 con 0x7ffa541575a0 2024-04-25T16:27:17.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa5a14d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5406f3a0 con 0x7ffa540b0860 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa53fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffa540b0860 0x7ffa54160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa53fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffa540b0860 0x7ffa54160e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35748/0 (socket says 172.21.15.94:35748) 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa53fff640 1 -- 172.21.15.94:0/3789119458 learned_addr learned my addr 172.21.15.94:0/3789119458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.457+0000 7ffa4bfff640 1 --1- 172.21.15.94:0/3789119458 >> v1:172.21.15.94:6789/0 conn(0x7ffa5406d580 0x7ffa5406d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42002/0 (socket says 172.21.15.94:42002) 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.458+0000 7ffa53fff640 1 -- 172.21.15.94:0/3789119458 >> v1:172.21.15.138:6789/0 conn(0x7ffa541575a0 legacy=0x7ffa54070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.458+0000 7ffa53fff640 1 -- 172.21.15.94:0/3789119458 >> v1:172.21.15.94:6789/0 conn(0x7ffa5406d580 legacy=0x7ffa5406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.458+0000 7ffa53fff640 1 -- 172.21.15.94:0/3789119458 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5406fbb0 con 0x7ffa540b0860 2024-04-25T16:27:17.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.458+0000 7ffa53fff640 1 --2- 172.21.15.94:0/3789119458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa540b0860 0x7ffa54160e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e50 tx=0x7ffa4c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c986f368cf9434e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:17.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.458+0000 7ffa52ffd640 1 -- 172.21.15.94:0/3789119458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c036070 con 0x7ffa540b0860 2024-04-25T16:27:17.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa52ffd640 1 -- 172.21.15.94:0/3789119458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c031600 con 0x7ffa540b0860 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa52ffd640 1 -- 172.21.15.94:0/3789119458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0319d0 con 0x7ffa540b0860 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 -- 172.21.15.94:0/3789119458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa540b0860 msgr2=0x7ffa54160e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/3789119458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa540b0860 0x7ffa54160e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e50 tx=0x7ffa4c02a760 comp rx=0 tx=0).stop 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 -- 172.21.15.94:0/3789119458 shutdown_connections 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/3789119458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa540b0860 0x7ffa54160e80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 -- 172.21.15.94:0/3789119458 >> 172.21.15.94:0/3789119458 conn(0x7ffa540d23b0 msgr2=0x7ffa54073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 -- 172.21.15.94:0/3789119458 shutdown_connections 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.459+0000 7ffa5a14d640 1 -- 172.21.15.94:0/3789119458 wait complete. 2024-04-25T16:27:17.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.461+0000 7ffa5a14d640 1 Processor -- start 2024-04-25T16:27:17.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.461+0000 7ffa5a14d640 1 -- start start 2024-04-25T16:27:17.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa5a14d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 0x7ffa54159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa5a14d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa540b0860 0x7ffa541f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa5a14d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa541575a0 0x7ffa54160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa5a14d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54071300 con 0x7ffa5406d580 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa5a14d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54070170 con 0x7ffa541575a0 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa5a14d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa541579b0 con 0x7ffa540b0860 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 0x7ffa54159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.462+0000 7ffa537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 0x7ffa54159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35752/0 (socket says 172.21.15.94:35752) 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 -- 172.21.15.94:0/1354511800 learned_addr learned my addr 172.21.15.94:0/1354511800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa4bfff640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa540b0860 0x7ffa541f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa53fff640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa541575a0 0x7ffa54160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 -- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa540b0860 msgr2=0x7ffa541f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa540b0860 0x7ffa541f6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 -- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa541575a0 msgr2=0x7ffa54160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa541575a0 0x7ffa54160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 -- 172.21.15.94:0/1354511800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa541e9ff0 con 0x7ffa5406d580 2024-04-25T16:27:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa53fff640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa541575a0 0x7ffa54160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa537fe640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 0x7ffa54159400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b3f0 tx=0x7ffa4400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44015020 con 0x7ffa5406d580 2024-04-25T16:27:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa44011830 con 0x7ffa5406d580 2024-04-25T16:27:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa541ea2d0 con 0x7ffa5406d580 2024-04-25T16:27:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44011c20 con 0x7ffa5406d580 2024-04-25T16:27:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.463+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541ea890 con 0x7ffa5406d580 2024-04-25T16:27:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.464+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa54075610 con 0x7ffa5406d580 2024-04-25T16:27:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.465+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa44018020 con 0x7ffa5406d580 2024-04-25T16:27:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.465+0000 7ffa50ff9640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ffa340744b0 0x7ffa340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.465+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa440951e0 con 0x7ffa5406d580 2024-04-25T16:27:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.465+0000 7ffa4bfff640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ffa340744b0 0x7ffa340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.466+0000 7ffa4bfff640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ffa340744b0 0x7ffa340769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa5415a060 tx=0x7ffa3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.470+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa54075610 con 0x7ffa5406d580 2024-04-25T16:27:17.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.646+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa5406a180 con 0x7ffa5406d580 2024-04-25T16:27:17.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.648+0000 7ffa50ff9640 1 -- 172.21.15.94:0/1354511800 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7ffa4401f030 con 0x7ffa5406d580 2024-04-25T16:27:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:17.649 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ffa340744b0 msgr2=0x7ffa340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ffa340744b0 0x7ffa340769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa5415a060 tx=0x7ffa3c009040 comp rx=0 tx=0).stop 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 msgr2=0x7ffa54159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 0x7ffa54159400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b3f0 tx=0x7ffa4400b8b0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 shutdown_connections 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ffa340744b0 0x7ffa340769c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa541575a0 0x7ffa54160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa540b0860 0x7ffa541f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 --2- 172.21.15.94:0/1354511800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa5406d580 0x7ffa54159400 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 >> 172.21.15.94:0/1354511800 conn(0x7ffa540d23b0 msgr2=0x7ffa541556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 shutdown_connections 2024-04-25T16:27:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.650+0000 7ffa5a14d640 1 -- 172.21.15.94:0/1354511800 wait complete. 2024-04-25T16:27:17.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.769+0000 7fdde41b7640 1 Processor -- start 2024-04-25T16:27:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.769+0000 7fdde41b7640 1 -- start start 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde41b7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdddc0b5230 0x7fdddc160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde41b7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdddc164d30 0x7fdddc06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde41b7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdddc0b5a30 con 0x7fdddc164210 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde41b7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc06ca10 con 0x7fdddc0b5230 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde41b7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc0d5610 con 0x7fdddc164d30 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1f53640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdddc0b5230 0x7fdddc160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1752640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdddc164210 0x7fdddc164620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42018/0 (socket says 172.21.15.94:42018) 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1f53640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdddc0b5230 0x7fdddc160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35754/0 (socket says 172.21.15.94:35754) 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1752640 1 -- 172.21.15.94:0/2175601896 learned_addr learned my addr 172.21.15.94:0/2175601896 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde2754640 1 --2- 172.21.15.94:0/2175601896 >> v2:172.21.15.138:3300/0 conn(0x7fdddc164d30 0x7fdddc06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1f53640 1 -- 172.21.15.94:0/2175601896 >> v2:172.21.15.138:3300/0 conn(0x7fdddc164d30 msgr2=0x7fdddc06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1f53640 1 --2- 172.21.15.94:0/2175601896 >> v2:172.21.15.138:3300/0 conn(0x7fdddc164d30 0x7fdddc06cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1f53640 1 -- 172.21.15.94:0/2175601896 >> v1:172.21.15.94:6789/0 conn(0x7fdddc164210 legacy=0x7fdddc164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde0f51640 1 -- 172.21.15.94:0/2175601896 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495278494 0 0) 0x7fddcc001110 con 0x7fdddc164210 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.771+0000 7fdde1f53640 1 -- 172.21.15.94:0/2175601896 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0d58f0 con 0x7fdddc0b5230 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.772+0000 7fdde1f53640 1 --2- 172.21.15.94:0/2175601896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc0b5230 0x7fdddc160790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b8da0 tx=0x7fddd802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a20e89dc4c48cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.772+0000 7fdde0f51640 1 -- 172.21.15.94:0/2175601896 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8037070 con 0x7fdddc0b5230 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.772+0000 7fdde0f51640 1 -- 172.21.15.94:0/2175601896 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd8032440 con 0x7fdddc0b5230 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.772+0000 7fdde0f51640 1 -- 172.21.15.94:0/2175601896 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd802ae60 con 0x7fdddc0b5230 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 -- 172.21.15.94:0/2175601896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc0b5230 msgr2=0x7fdddc160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 --2- 172.21.15.94:0/2175601896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc0b5230 0x7fdddc160790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b8da0 tx=0x7fddd802a110 comp rx=0 tx=0).stop 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 -- 172.21.15.94:0/2175601896 shutdown_connections 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 --2- 172.21.15.94:0/2175601896 >> v2:172.21.15.138:3300/0 conn(0x7fdddc164d30 0x7fdddc06cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 --2- 172.21.15.94:0/2175601896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc0b5230 0x7fdddc160790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 -- 172.21.15.94:0/2175601896 >> 172.21.15.94:0/2175601896 conn(0x7fdddc0d3570 msgr2=0x7fdddc0b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 -- 172.21.15.94:0/2175601896 shutdown_connections 2024-04-25T16:27:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.773+0000 7fdde41b7640 1 -- 172.21.15.94:0/2175601896 wait complete. 2024-04-25T16:27:17.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.774+0000 7fdde41b7640 1 Processor -- start 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.774+0000 7fdde41b7640 1 -- start start 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdddc0b5230 0x7fdddc1eab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdddc164210 0x7fdddc15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 0x7fdddc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0b9870 con 0x7fdddc164d30 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0b96f0 con 0x7fdddc0b5230 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc06d3e0 con 0x7fdddc164210 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 0x7fdddc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 0x7fdddc0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35764/0 (socket says 172.21.15.94:35764) 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 -- 172.21.15.94:0/180376873 learned_addr learned my addr 172.21.15.94:0/180376873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde1f53640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdddc0b5230 0x7fdddc1eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde1752640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdddc164210 0x7fdddc15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 -- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdddc164210 msgr2=0x7fdddc15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdddc164210 0x7fdddc15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 -- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdddc0b5230 msgr2=0x7fdddc1eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdddc0b5230 0x7fdddc1eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 -- 172.21.15.94:0/180376873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0ccaf0 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde2754640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 0x7fdddc0b40f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009d30 tx=0x7fddd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4018020 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd4014480 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc0ccdd0 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4014870 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.776+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc0ca500 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.778+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc06a180 con 0x7fdddc164d30 2024-04-25T16:27:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.778+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fddd4005d10 con 0x7fdddc164d30 2024-04-25T16:27:17.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.779+0000 7fddd27fc640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fddc00744e0 0x7fddc00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:17.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.779+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fddd40961b0 con 0x7fdddc164d30 2024-04-25T16:27:17.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.779+0000 7fdde1f53640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fddc00744e0 0x7fddc00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:17.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.779+0000 7fdde1f53640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fddc00744e0 0x7fddc00769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddd8034000 tx=0x7fddd802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.783+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddd4022030 con 0x7fdddc164d30 2024-04-25T16:27:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.960+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdddc0ccfa0 con 0x7fdddc164d30 2024-04-25T16:27:17.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.962+0000 7fddd27fc640 1 -- 172.21.15.94:0/180376873 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fddd4010040 con 0x7fdddc164d30 2024-04-25T16:27:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:17.963 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fddc00744e0 msgr2=0x7fddc00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fddc00744e0 0x7fddc00769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddd8034000 tx=0x7fddd802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 msgr2=0x7fdddc0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 0x7fdddc0b40f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009d30 tx=0x7fddd400c490 comp rx=0 tx=0).stop 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 shutdown_connections 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fddc00744e0 0x7fddc00769f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdddc164d30 0x7fdddc0b40f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdddc164210 0x7fdddc15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 --2- 172.21.15.94:0/180376873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdddc0b5230 0x7fdddc1eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 >> 172.21.15.94:0/180376873 conn(0x7fdddc0d3570 msgr2=0x7fdddc0bb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 shutdown_connections 2024-04-25T16:27:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:17.964+0000 7fdde41b7640 1 -- 172.21.15.94:0/180376873 wait complete. 2024-04-25T16:27:17.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T16:27:18.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.085+0000 7ff412205640 1 Processor -- start 2024-04-25T16:27:18.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.085+0000 7ff412205640 1 -- start start 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff412205640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff40c0b4c70 0x7ff40c160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff412205640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff40c0d4a20 0x7ff40c0d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff412205640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff40c161930 con 0x7ff40c0b5710 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff412205640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff40c161c10 con 0x7ff40c0d4a20 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff412205640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff40c164d10 con 0x7ff40c0b4c70 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff40c0d4a20 0x7ff40c0d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff40c0d4a20 0x7ff40c0d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35776/0 (socket says 172.21.15.94:35776) 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 -- 172.21.15.94:0/1180040398 learned_addr learned my addr 172.21.15.94:0/1180040398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40affd640 1 --2- 172.21.15.94:0/1180040398 >> v2:172.21.15.138:3300/0 conn(0x7ff40c0b4c70 0x7ff40c160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 -- 172.21.15.94:0/1180040398 >> v2:172.21.15.138:3300/0 conn(0x7ff40c0b4c70 msgr2=0x7ff40c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:18.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 --2- 172.21.15.94:0/1180040398 >> v2:172.21.15.138:3300/0 conn(0x7ff40c0b4c70 0x7ff40c160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 -- 172.21.15.94:0/1180040398 >> v1:172.21.15.138:6789/0 conn(0x7ff40c0b5710 legacy=0x7ff40c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:18.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40bfff640 1 -- 172.21.15.94:0/1180040398 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c164ff0 con 0x7ff40c0d4a20 2024-04-25T16:27:18.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.087+0000 7ff40affd640 1 --2- 172.21.15.94:0/1180040398 >> v2:172.21.15.138:3300/0 conn(0x7ff40c0b4c70 0x7ff40c160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.088+0000 7ff40bfff640 1 --2- 172.21.15.94:0/1180040398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0d4a20 0x7ff40c0d4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0d72a0 tx=0x7ff40402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7329046549877d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.088+0000 7ff40a7fc640 1 -- 172.21.15.94:0/1180040398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404036070 con 0x7ff40c0d4a20 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.088+0000 7ff40a7fc640 1 -- 172.21.15.94:0/1180040398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff404031600 con 0x7ff40c0d4a20 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.088+0000 7ff40a7fc640 1 -- 172.21.15.94:0/1180040398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040319d0 con 0x7ff40c0d4a20 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 -- 172.21.15.94:0/1180040398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0d4a20 msgr2=0x7ff40c0d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 --2- 172.21.15.94:0/1180040398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0d4a20 0x7ff40c0d4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0d72a0 tx=0x7ff40402a610 comp rx=0 tx=0).stop 2024-04-25T16:27:18.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 -- 172.21.15.94:0/1180040398 shutdown_connections 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 --2- 172.21.15.94:0/1180040398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0d4a20 0x7ff40c0d4e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 --2- 172.21.15.94:0/1180040398 >> v2:172.21.15.138:3300/0 conn(0x7ff40c0b4c70 0x7ff40c160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 -- 172.21.15.94:0/1180040398 >> 172.21.15.94:0/1180040398 conn(0x7ff40c0d6930 msgr2=0x7ff40c0b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 -- 172.21.15.94:0/1180040398 shutdown_connections 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.089+0000 7ff412205640 1 -- 172.21.15.94:0/1180040398 wait complete. 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.090+0000 7ff412205640 1 Processor -- start 2024-04-25T16:27:18.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.091+0000 7ff412205640 1 -- start start 2024-04-25T16:27:18.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff412205640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 0x7ff40c15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff412205640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40c0b5710 0x7ff40c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff412205640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff40c0d4a20 0x7ff40c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff412205640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c161530 con 0x7ff40c0b4c70 2024-04-25T16:27:18.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff412205640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c0b51c0 con 0x7ff40c0b5710 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff412205640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff40c1616b0 con 0x7ff40c0d4a20 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 0x7ff40c15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 0x7ff40c15cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35780/0 (socket says 172.21.15.94:35780) 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 -- 172.21.15.94:0/3340347976 learned_addr learned my addr 172.21.15.94:0/3340347976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40affd640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40c0b5710 0x7ff40c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 -- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff40c0d4a20 msgr2=0x7ff40c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40bfff640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff40c0d4a20 0x7ff40c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff40c0d4a20 0x7ff40c0cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 -- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40c0b5710 msgr2=0x7ff40c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40c0b5710 0x7ff40c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40b7fe640 1 -- 172.21.15.94:0/3340347976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c0cd740 con 0x7ff40c0b4c70 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.092+0000 7ff40affd640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40c0b5710 0x7ff40c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.093+0000 7ff40b7fe640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 0x7ff40c15cd80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c470 tx=0x7ff3fc00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.093+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0052d0 con 0x7ff40c0b4c70 2024-04-25T16:27:18.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.093+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc005490 con 0x7ff40c0b4c70 2024-04-25T16:27:18.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.093+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc007260 con 0x7ff40c0b4c70 2024-04-25T16:27:18.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.093+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c0cc130 con 0x7ff40c0b4c70 2024-04-25T16:27:18.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.093+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c0cc920 con 0x7ff40c0b4c70 2024-04-25T16:27:18.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.094+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c0ccaf0 con 0x7ff40c0b4c70 2024-04-25T16:27:18.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.095+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3fc007400 con 0x7ff40c0b4c70 2024-04-25T16:27:18.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.095+0000 7ff3f3fff640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff3f80744b0 0x7ff3f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.095+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff3fc094990 con 0x7ff40c0b4c70 2024-04-25T16:27:18.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.095+0000 7ff40affd640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff3f80744b0 0x7ff3f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.095+0000 7ff40affd640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff3f80744b0 0x7ff3f80769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff40c06dd30 tx=0x7ff400008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:18.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.100+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3fc05ede0 con 0x7ff40c0b4c70 2024-04-25T16:27:18.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.311+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff40c06a180 con 0x7ff40c0b4c70 2024-04-25T16:27:18.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.625+0000 7ff3f3fff640 1 -- 172.21.15.94:0/3340347976 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7ff3fc05e6e0 con 0x7ff40c0b4c70 2024-04-25T16:27:18.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff3f80744b0 msgr2=0x7ff3f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff3f80744b0 0x7ff3f80769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff40c06dd30 tx=0x7ff400008040 comp rx=0 tx=0).stop 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 msgr2=0x7ff40c15cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 0x7ff40c15cd80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c470 tx=0x7ff3fc00c930 comp rx=0 tx=0).stop 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 shutdown_connections 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff3f80744b0 0x7ff3f80769c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff40c0d4a20 0x7ff40c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40c0b5710 0x7ff40c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 --2- 172.21.15.94:0/3340347976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c0b4c70 0x7ff40c15cd80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 >> 172.21.15.94:0/3340347976 conn(0x7ff40c0d6930 msgr2=0x7ff40c06cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 shutdown_connections 2024-04-25T16:27:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.627+0000 7ff412205640 1 -- 172.21.15.94:0/3340347976 wait complete. 2024-04-25T16:27:18.639 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T16:27:18.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T16:27:18.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.748+0000 7f84bb5f3640 1 Processor -- start 2024-04-25T16:27:18.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.748+0000 7f84bb5f3640 1 -- start start 2024-04-25T16:27:18.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84bb5f3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84b40b5420 0x7f84b409c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84bb5f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b4162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84bb5f3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f84b4163280 0x7f84b4163690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84bb5f3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f84b4163bd0 con 0x7f84b4163280 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84bb5f3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f84b40b34b0 con 0x7f84b40b5420 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84bb5f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84b40d2270 con 0x7f84b409c7f0 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f84b4163280 0x7f84b4163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f84b4163280 0x7f84b4163690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35796/0 (socket says 172.21.15.94:35796) 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 -- 172.21.15.94:0/1588594388 learned_addr learned my addr 172.21.15.94:0/1588594388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b938f640 1 --2- 172.21.15.94:0/1588594388 >> v2:172.21.15.138:3300/0 conn(0x7f84b40b5420 0x7f84b409c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b8b8e640 1 --2- 172.21.15.94:0/1588594388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b4162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 -- 172.21.15.94:0/1588594388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 msgr2=0x7f84b4162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 --2- 172.21.15.94:0/1588594388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b4162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 -- 172.21.15.94:0/1588594388 >> v2:172.21.15.138:3300/0 conn(0x7f84b40b5420 msgr2=0x7f84b409c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:18.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 --2- 172.21.15.94:0/1588594388 >> v2:172.21.15.138:3300/0 conn(0x7f84b40b5420 0x7f84b409c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.750+0000 7f84b9b90640 1 -- 172.21.15.94:0/1588594388 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b40d1f90 con 0x7f84b4163280 2024-04-25T16:27:18.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.751+0000 7f84b9b90640 1 --2- 172.21.15.94:0/1588594388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b4163280 0x7f84b4163690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84b4063b40 tx=0x7f84ac02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0aa8521937a143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:18.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.751+0000 7f84abfff640 1 -- 172.21.15.94:0/1588594388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac036070 con 0x7f84b4163280 2024-04-25T16:27:18.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.751+0000 7f84abfff640 1 -- 172.21.15.94:0/1588594388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ac031600 con 0x7f84b4163280 2024-04-25T16:27:18.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.751+0000 7f84abfff640 1 -- 172.21.15.94:0/1588594388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0319d0 con 0x7f84b4163280 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 -- 172.21.15.94:0/1588594388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b4163280 msgr2=0x7f84b4163690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/1588594388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b4163280 0x7f84b4163690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84b4063b40 tx=0x7f84ac02a610 comp rx=0 tx=0).stop 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 -- 172.21.15.94:0/1588594388 shutdown_connections 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/1588594388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b4163280 0x7f84b4163690 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/1588594388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b4162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/1588594388 >> v2:172.21.15.138:3300/0 conn(0x7f84b40b5420 0x7f84b409c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 -- 172.21.15.94:0/1588594388 >> 172.21.15.94:0/1588594388 conn(0x7f84b40b5b20 msgr2=0x7f84b40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 -- 172.21.15.94:0/1588594388 shutdown_connections 2024-04-25T16:27:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.752+0000 7f84bb5f3640 1 -- 172.21.15.94:0/1588594388 wait complete. 2024-04-25T16:27:18.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.754+0000 7f84bb5f3640 1 Processor -- start 2024-04-25T16:27:18.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.754+0000 7f84bb5f3640 1 -- start start 2024-04-25T16:27:18.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84bb5f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84bb5f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 0x7f84b40c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84bb5f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84b4163280 0x7f84b40cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84bb5f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84b40cfed0 con 0x7f84b40b5420 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84bb5f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84b409d430 con 0x7f84b4163280 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84bb5f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84b40b0c20 con 0x7f84b409c7f0 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 0x7f84b40c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 0x7f84b40c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35810/0 (socket says 172.21.15.94:35810) 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 -- 172.21.15.94:0/3348125530 learned_addr learned my addr 172.21.15.94:0/3348125530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b938f640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b9b90640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84b4163280 0x7f84b40cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 -- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 msgr2=0x7f84b40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 -- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84b4163280 msgr2=0x7f84b40cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84b4163280 0x7f84b40cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b8b8e640 1 -- 172.21.15.94:0/3348125530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b40cdd60 con 0x7f84b40b5420 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.755+0000 7f84b9b90640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84b4163280 0x7f84b40cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.756+0000 7f84b8b8e640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 0x7f84b40c8350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e870 tx=0x7f84a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.756+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4017070 con 0x7f84b40b5420 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.756+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a400ee60 con 0x7f84b40b5420 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.756+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b406e430 con 0x7f84b40b5420 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.756+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400cb20 con 0x7f84b40b5420 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.756+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41f72e0 con 0x7f84b40b5420 2024-04-25T16:27:18.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.757+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b406a180 con 0x7f84b40b5420 2024-04-25T16:27:18.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.758+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f84a401a070 con 0x7f84b40b5420 2024-04-25T16:27:18.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.758+0000 7f84a9ffb640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f84980744e0 0x7f84980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:18.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.758+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f84a4095270 con 0x7f84b40b5420 2024-04-25T16:27:18.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.758+0000 7f84b938f640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f84980744e0 0x7f84980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:18.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.759+0000 7f84b938f640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f84980744e0 0x7f84980769f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009aa0 tx=0x7f84b0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.763+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84a405f070 con 0x7f84b40b5420 2024-04-25T16:27:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:18.973+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f84b409ce20 con 0x7f84b40b5420 2024-04-25T16:27:19.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.635+0000 7f84a9ffb640 1 -- 172.21.15.94:0/3348125530 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f84a4063370 con 0x7f84b40b5420 2024-04-25T16:27:19.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f84980744e0 msgr2=0x7f84980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f84980744e0 0x7f84980769f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009aa0 tx=0x7f84b0009190 comp rx=0 tx=0).stop 2024-04-25T16:27:19.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 msgr2=0x7f84b40c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 0x7f84b40c8350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e870 tx=0x7f84a400ed30 comp rx=0 tx=0).stop 2024-04-25T16:27:19.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 shutdown_connections 2024-04-25T16:27:19.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f84980744e0 0x7f84980769f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84b4163280 0x7f84b40cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b40b5420 0x7f84b40c8350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 --2- 172.21.15.94:0/3348125530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84b409c7f0 0x7f84b40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 >> 172.21.15.94:0/3348125530 conn(0x7f84b40b5b20 msgr2=0x7f84b40b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:19.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 shutdown_connections 2024-04-25T16:27:19.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.637+0000 7f84bb5f3640 1 -- 172.21.15.94:0/3348125530 wait complete. 2024-04-25T16:27:19.650 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T16:27:19.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:27:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.758+0000 7f2ee0284640 1 Processor -- start 2024-04-25T16:27:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.758+0000 7f2ee0284640 1 -- start start 2024-04-25T16:27:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ee0284640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80d11b0 0x7f2ed80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ee0284640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ed808e080 0x7f2ed80cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ee0284640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed80d24a0 con 0x7f2ed80d2090 2024-04-25T16:27:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ee0284640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed80d2720 con 0x7f2ed808e080 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ee0284640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed80d2a00 con 0x7f2ed80d11b0 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ede020640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2ed80d2090 0x7f2ed80d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42024/0 (socket says 172.21.15.94:42024) 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ede020640 1 -- 172.21.15.94:0/2809427694 learned_addr learned my addr 172.21.15.94:0/2809427694 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2edd81f640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80d11b0 0x7f2ed80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2ede821640 1 --2- 172.21.15.94:0/2809427694 >> v2:172.21.15.138:3300/0 conn(0x7f2ed808e080 0x7f2ed80cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2edd01e640 1 -- 172.21.15.94:0/2809427694 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335193397 0 0) 0x7f2ed4001290 con 0x7f2ed80d2090 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.760+0000 7f2edd01e640 1 -- 172.21.15.94:0/2809427694 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ec4003520 con 0x7f2ed80d2090 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.761+0000 7f2ede821640 1 -- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80d11b0 msgr2=0x7f2ed80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.761+0000 7f2ede821640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80d11b0 0x7f2ed80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.761+0000 7f2ede821640 1 -- 172.21.15.94:0/2809427694 >> v1:172.21.15.94:6789/0 conn(0x7f2ed80d2090 legacy=0x7f2ed80d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.761+0000 7f2ede821640 1 -- 172.21.15.94:0/2809427694 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80d2ce0 con 0x7f2ed808e080 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.761+0000 7f2edd81f640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80d11b0 0x7f2ed80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.761+0000 7f2ede821640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed808e080 0x7f2ed80cf2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009d90 tx=0x7f2ed002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4c038f1dce4558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2edd01e640 1 -- 172.21.15.94:0/2809427694 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0031600 con 0x7f2ed808e080 2024-04-25T16:27:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2edd01e640 1 -- 172.21.15.94:0/2809427694 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed00317a0 con 0x7f2ed808e080 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2edd01e640 1 -- 172.21.15.94:0/2809427694 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0031b50 con 0x7f2ed808e080 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 -- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed808e080 msgr2=0x7f2ed80cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed808e080 0x7f2ed80cf2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009d90 tx=0x7f2ed002a5d0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 -- 172.21.15.94:0/2809427694 shutdown_connections 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed808e080 0x7f2ed80cf2e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2809427694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80d11b0 0x7f2ed80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 -- 172.21.15.94:0/2809427694 >> 172.21.15.94:0/2809427694 conn(0x7f2ed80d4090 msgr2=0x7f2ed80d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 -- 172.21.15.94:0/2809427694 shutdown_connections 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.762+0000 7f2ee0284640 1 -- 172.21.15.94:0/2809427694 wait complete. 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.764+0000 7f2ee0284640 1 Processor -- start 2024-04-25T16:27:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.764+0000 7f2ee0284640 1 -- start start 2024-04-25T16:27:19.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.765+0000 7f2ee0284640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed808e080 0x7f2ed80b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:19.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.765+0000 7f2ee0284640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 0x7f2ed80c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:19.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.765+0000 7f2ee0284640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8156e20 0x7f2ed8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:19.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.765+0000 7f2ee0284640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80ced50 con 0x7f2ed80d2090 2024-04-25T16:27:19.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.765+0000 7f2ee0284640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed808e490 con 0x7f2ed8156e20 2024-04-25T16:27:19.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.765+0000 7f2ee0284640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed80b4470 con 0x7f2ed808e080 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 0x7f2ed80c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 0x7f2ed80c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35818/0 (socket says 172.21.15.94:35818) 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 -- 172.21.15.94:0/2966064665 learned_addr learned my addr 172.21.15.94:0/2966064665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2ede020640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed808e080 0x7f2ed80b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2ede821640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8156e20 0x7f2ed8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 -- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed808e080 msgr2=0x7f2ed80b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed808e080 0x7f2ed80b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 -- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8156e20 msgr2=0x7f2ed8153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8156e20 0x7f2ed8153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 -- 172.21.15.94:0/2966064665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80ca1c0 con 0x7f2ed80d2090 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2ede020640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed808e080 0x7f2ed80b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:19.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2edd81f640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 0x7f2ed80c9c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800e900 tx=0x7f2ec800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8017070 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec800ca00 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed80b2020 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800cdf0 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.766+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed80b22c0 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.767+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed806a180 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.768+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ec801a070 con 0x7f2ed80d2090 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.768+0000 7f2eceffd640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ebc0744e0 0x7f2ebc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.769+0000 7f2ede020640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ebc0744e0 0x7f2ebc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:19.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.769+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2ec8094fb0 con 0x7f2ed80d2090 2024-04-25T16:27:19.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.769+0000 7f2ede020640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ebc0744e0 0x7f2ebc0769f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001530 tx=0x7f2ed4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:19.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.773+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ed80ca1c0 con 0x7f2ed80d2090 2024-04-25T16:27:19.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.984+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ed80b4020 con 0x7f2ed80d2090 2024-04-25T16:27:19.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.985+0000 7f2eceffd640 1 -- 172.21.15.94:0/2966064665 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f2ec805f490 con 0x7f2ed80d2090 2024-04-25T16:27:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:19.985 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:18.975101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:27:19.986 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:27:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ebc0744e0 msgr2=0x7f2ebc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ebc0744e0 0x7f2ebc0769f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001530 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2024-04-25T16:27:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 msgr2=0x7f2ed80c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 0x7f2ed80c9c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800e900 tx=0x7f2ec800edc0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 shutdown_connections 2024-04-25T16:27:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ebc0744e0 0x7f2ebc0769f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8156e20 0x7f2ed8153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ed80d2090 0x7f2ed80c9c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 --2- 172.21.15.94:0/2966064665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed808e080 0x7f2ed80b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.986+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 >> 172.21.15.94:0/2966064665 conn(0x7f2ed80d4090 msgr2=0x7f2ed80b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.987+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 shutdown_connections 2024-04-25T16:27:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:19.987+0000 7f2ee0284640 1 -- 172.21.15.94:0/2966064665 wait complete. 2024-04-25T16:27:19.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.104+0000 7f4bb8497640 1 Processor -- start 2024-04-25T16:27:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.104+0000 7f4bb8497640 1 -- start start 2024-04-25T16:27:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.105+0000 7f4bb8497640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00b37c0 0x7f4bb00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.105+0000 7f4bb8497640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4bb00cda90 0x7f4bb00d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.105+0000 7f4bb8497640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb0160db0 con 0x7f4bb00d2bc0 2024-04-25T16:27:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.105+0000 7f4bb8497640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb00ce1b0 con 0x7f4bb00cda90 2024-04-25T16:27:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.105+0000 7f4bb8497640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb00ce490 con 0x7f4bb00b37c0 2024-04-25T16:27:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4bb00cda90 0x7f4bb00d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4bb00cda90 0x7f4bb00d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35826/0 (socket says 172.21.15.94:35826) 2024-04-25T16:27:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 -- 172.21.15.94:0/2891711010 learned_addr learned my addr 172.21.15.94:0/2891711010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb6233640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00b37c0 0x7f4bb00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 -- 172.21.15.94:0/2891711010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00b37c0 msgr2=0x7f4bb00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00b37c0 0x7f4bb00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 -- 172.21.15.94:0/2891711010 >> v1:172.21.15.138:6789/0 conn(0x7f4bb00d2bc0 legacy=0x7f4bb00d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:20.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb5a32640 1 -- 172.21.15.94:0/2891711010 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb00ce770 con 0x7f4bb00cda90 2024-04-25T16:27:20.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.106+0000 7f4bb6233640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00b37c0 0x7f4bb00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:20.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.107+0000 7f4bb5a32640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00cda90 0x7f4bb00d2680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009870 tx=0x7f4ba002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dde363b77fcd20c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.107+0000 7f4bb5231640 1 -- 172.21.15.94:0/2891711010 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0037070 con 0x7f4bb00cda90 2024-04-25T16:27:20.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.107+0000 7f4bb5231640 1 -- 172.21.15.94:0/2891711010 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba0032440 con 0x7f4bb00cda90 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.107+0000 7f4bb5231640 1 -- 172.21.15.94:0/2891711010 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0032810 con 0x7f4bb00cda90 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.107+0000 7f4bb8497640 1 -- 172.21.15.94:0/2891711010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00cda90 msgr2=0x7f4bb00d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.107+0000 7f4bb8497640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00cda90 0x7f4bb00d2680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009870 tx=0x7f4ba002a180 comp rx=0 tx=0).stop 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.108+0000 7f4bb8497640 1 -- 172.21.15.94:0/2891711010 shutdown_connections 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.108+0000 7f4bb8497640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00cda90 0x7f4bb00d2680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.108+0000 7f4bb8497640 1 --2- 172.21.15.94:0/2891711010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00b37c0 0x7f4bb00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.108+0000 7f4bb8497640 1 -- 172.21.15.94:0/2891711010 >> 172.21.15.94:0/2891711010 conn(0x7f4bb00b7bc0 msgr2=0x7f4bb00b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.108+0000 7f4bb8497640 1 -- 172.21.15.94:0/2891711010 shutdown_connections 2024-04-25T16:27:20.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.108+0000 7f4bb8497640 1 -- 172.21.15.94:0/2891711010 wait complete. 2024-04-25T16:27:20.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.109+0000 7f4bb8497640 1 Processor -- start 2024-04-25T16:27:20.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.109+0000 7f4bb8497640 1 -- start start 2024-04-25T16:27:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 0x7f4bb014e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00cda90 0x7f4bb015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb00d2bc0 0x7f4bb015d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb00d20f0 con 0x7f4bb00b37c0 2024-04-25T16:27:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb00cdea0 con 0x7f4bb00d2bc0 2024-04-25T16:27:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb006f7f0 con 0x7f4bb00cda90 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 0x7f4bb014e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 0x7f4bb014e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35828/0 (socket says 172.21.15.94:35828) 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 -- 172.21.15.94:0/727376529 learned_addr learned my addr 172.21.15.94:0/727376529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6a34640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb00d2bc0 0x7f4bb015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb5a32640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00cda90 0x7f4bb015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 -- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00cda90 msgr2=0x7f4bb015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00cda90 0x7f4bb015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 -- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb00d2bc0 msgr2=0x7f4bb015d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb00d2bc0 0x7f4bb015d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 -- 172.21.15.94:0/727376529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb006d8b0 con 0x7f4bb00b37c0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6a34640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb00d2bc0 0x7f4bb015d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb5a32640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00cda90 0x7f4bb015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb6233640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 0x7f4bb014e350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00cde0 tx=0x7f4bac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00f680 con 0x7f4bb00b37c0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac00f820 con 0x7f4bb00b37c0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb014e890 con 0x7f4bb00b37c0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00fc10 con 0x7f4bb00b37c0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.111+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb006db20 con 0x7f4bb00b37c0 2024-04-25T16:27:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.112+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb006a180 con 0x7f4bb00b37c0 2024-04-25T16:27:20.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.113+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4bac018400 con 0x7f4bb00b37c0 2024-04-25T16:27:20.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.113+0000 7f4ba6ffd640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b940744b0 0x7f4b940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.113+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4bac096430 con 0x7f4bb00b37c0 2024-04-25T16:27:20.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.113+0000 7f4bb5a32640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b940744b0 0x7f4b940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.114+0000 7f4bb5a32640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b940744b0 0x7f4b940769c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0034000 tx=0x7f4ba002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.118+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bb014e890 con 0x7f4bb00b37c0 2024-04-25T16:27:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.292+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bb01606c0 con 0x7f4bb00b37c0 2024-04-25T16:27:20.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.294+0000 7f4ba6ffd640 1 -- 172.21.15.94:0/727376529 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7f4bac014030 con 0x7f4bb00b37c0 2024-04-25T16:27:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:20.295 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b940744b0 msgr2=0x7f4b940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b940744b0 0x7f4b940769c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0034000 tx=0x7f4ba002e000 comp rx=0 tx=0).stop 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 msgr2=0x7f4bb014e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 0x7f4bb014e350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00cde0 tx=0x7f4bac00a650 comp rx=0 tx=0).stop 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 shutdown_connections 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b940744b0 0x7f4b940769c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb00d2bc0 0x7f4bb015d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bb00cda90 0x7f4bb015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 --2- 172.21.15.94:0/727376529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb00b37c0 0x7f4bb014e350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 >> 172.21.15.94:0/727376529 conn(0x7f4bb00b7bc0 msgr2=0x7f4bb00b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 shutdown_connections 2024-04-25T16:27:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.296+0000 7f4bb8497640 1 -- 172.21.15.94:0/727376529 wait complete. 2024-04-25T16:27:20.308 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T16:27:20.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:20.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.415+0000 7fa62a3a8640 1 Processor -- start 2024-04-25T16:27:20.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.415+0000 7fa62a3a8640 1 -- start start 2024-04-25T16:27:20.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa62a3a8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa62415e380 0x7fa6240d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa62a3a8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240d2bb0 con 0x7fa624161790 2024-04-25T16:27:20.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa62a3a8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240d2e90 con 0x7fa62409c0c0 2024-04-25T16:27:20.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa62a3a8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240d3450 con 0x7fa62415e380 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa6237fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa624161790 0x7fa62409b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42042/0 (socket says 172.21.15.94:42042) 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa6237fe640 1 -- 172.21.15.94:0/1206330488 learned_addr learned my addr 172.21.15.94:0/1206330488 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa623fff640 1 --2- 172.21.15.94:0/1206330488 >> v2:172.21.15.138:3300/0 conn(0x7fa62415e380 0x7fa6240d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa622ffd640 1 -- 172.21.15.94:0/1206330488 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883341941 0 0) 0x7fa6240d2bb0 con 0x7fa624161790 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.416+0000 7fa622ffd640 1 -- 172.21.15.94:0/1206330488 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa60c003520 con 0x7fa624161790 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.417+0000 7fa622ffd640 1 -- 172.21.15.94:0/1206330488 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 94510544 0 0) 0x7fa60c003520 con 0x7fa624161790 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.417+0000 7fa623fff640 1 -- 172.21.15.94:0/1206330488 >> v1:172.21.15.138:6789/0 conn(0x7fa62409c0c0 legacy=0x7fa6240b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.417+0000 7fa623fff640 1 -- 172.21.15.94:0/1206330488 >> v1:172.21.15.94:6789/0 conn(0x7fa624161790 legacy=0x7fa62409b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.417+0000 7fa623fff640 1 -- 172.21.15.94:0/1206330488 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240d3170 con 0x7fa62415e380 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa623fff640 1 --2- 172.21.15.94:0/1206330488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62415e380 0x7fa6240d2520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6140097a0 tx=0x7fa61402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d072a558ebda4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa622ffd640 1 -- 172.21.15.94:0/1206330488 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614037070 con 0x7fa62415e380 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa622ffd640 1 -- 172.21.15.94:0/1206330488 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61402abc0 con 0x7fa62415e380 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa622ffd640 1 -- 172.21.15.94:0/1206330488 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618001670 con 0x7fa62415e380 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa62a3a8640 1 -- 172.21.15.94:0/1206330488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62415e380 msgr2=0x7fa6240d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/1206330488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62415e380 0x7fa6240d2520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6140097a0 tx=0x7fa61402a130 comp rx=0 tx=0).stop 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa62a3a8640 1 -- 172.21.15.94:0/1206330488 shutdown_connections 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/1206330488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62415e380 0x7fa6240d2520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa62a3a8640 1 -- 172.21.15.94:0/1206330488 >> 172.21.15.94:0/1206330488 conn(0x7fa624074070 msgr2=0x7fa6240b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.418+0000 7fa62a3a8640 1 -- 172.21.15.94:0/1206330488 shutdown_connections 2024-04-25T16:27:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.419+0000 7fa62a3a8640 1 -- 172.21.15.94:0/1206330488 wait complete. 2024-04-25T16:27:20.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.420+0000 7fa62a3a8640 1 Processor -- start 2024-04-25T16:27:20.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.420+0000 7fa62a3a8640 1 -- start start 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa62a3a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62409c0c0 0x7fa62414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa62a3a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 0x7fa624161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa62a3a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa624161790 0x7fa624162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa62a3a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240b3090 con 0x7fa62415e380 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa62a3a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa624161ba0 con 0x7fa62409c0c0 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa62a3a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240b3210 con 0x7fa624161790 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 0x7fa624161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 0x7fa624161d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35834/0 (socket says 172.21.15.94:35834) 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 -- 172.21.15.94:0/3179257542 learned_addr learned my addr 172.21.15.94:0/3179257542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa623fff640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62409c0c0 0x7fa62414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa628945640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa624161790 0x7fa624162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 -- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa624161790 msgr2=0x7fa624162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa624161790 0x7fa624162260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 -- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62409c0c0 msgr2=0x7fa62414da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62409c0c0 0x7fa62414da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 -- 172.21.15.94:0/3179257542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62414df90 con 0x7fa62415e380 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa6237fe640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 0x7fa624161d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c580 tx=0x7fa61800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.422+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618017070 con 0x7fa62415e380 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.423+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa618005ce0 con 0x7fa62415e380 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.423+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6241f73e0 con 0x7fa62415e380 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.423+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618007330 con 0x7fa62415e380 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.423+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6241f79a0 con 0x7fa62415e380 2024-04-25T16:27:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.423+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62406a180 con 0x7fa62415e380 2024-04-25T16:27:20.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.424+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa61801a070 con 0x7fa62415e380 2024-04-25T16:27:20.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.424+0000 7fa620ff9640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa6100744b0 0x7fa6100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.425+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa618094920 con 0x7fa62415e380 2024-04-25T16:27:20.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.425+0000 7fa623fff640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa6100744b0 0x7fa6100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.425+0000 7fa623fff640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa6100744b0 0x7fa6100769c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa614034000 tx=0x7fa61402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.429+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa61805e7c0 con 0x7fa62415e380 2024-04-25T16:27:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.608+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6241607a0 con 0x7fa6100744b0 2024-04-25T16:27:20.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.617+0000 7fa620ff9640 1 -- 172.21.15.94:0/3179257542 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7fa6241607a0 con 0x7fa6100744b0 2024-04-25T16:27:20.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:20.623 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa6100744b0 msgr2=0x7fa6100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa6100744b0 0x7fa6100769c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa614034000 tx=0x7fa61402e000 comp rx=0 tx=0).stop 2024-04-25T16:27:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 msgr2=0x7fa624161d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 0x7fa624161d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c580 tx=0x7fa61800ca40 comp rx=0 tx=0).stop 2024-04-25T16:27:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 shutdown_connections 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa6100744b0 0x7fa6100769c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa624161790 0x7fa624162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa62415e380 0x7fa624161d20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 --2- 172.21.15.94:0/3179257542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa62409c0c0 0x7fa62414da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 >> 172.21.15.94:0/3179257542 conn(0x7fa624074070 msgr2=0x7fa6240705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 shutdown_connections 2024-04-25T16:27:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.620+0000 7fa62a3a8640 1 -- 172.21.15.94:0/3179257542 wait complete. 2024-04-25T16:27:20.632 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T16:27:19.302432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236072,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738648,"statfs":{"total":773094113280,"available":772852375552,"internally_reserved":0,"allocated":3350528,"data_stored":2020472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003159"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901527+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:02.901527+0000","last_peered":"2024-04-25T16:27:02.901527+0000","last_clean":"2024-04-25T16:27:02.901527+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:02.901527+0000","last_undegraded":"2024-04-25T16:27:02.901527+0000","last_fullsized":"2024-04-25T16:27:02.901527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584607+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:01.584607+0000","last_peered":"2024-04-25T16:27:01.584607+0000","last_clean":"2024-04-25T16:27:01.584607+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:01.584607+0000","last_undegraded":"2024-04-25T16:27:01.584607+0000","last_fullsized":"2024-04-25T16:27:01.584607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584101+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:01.584101+0000","last_peered":"2024-04-25T16:27:01.584101+0000","last_clean":"2024-04-25T16:27:01.584101+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:01.584101+0000","last_undegraded":"2024-04-25T16:27:01.584101+0000","last_fullsized":"2024-04-25T16:27:01.584101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584845+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:01.584845+0000","last_peered":"2024-04-25T16:27:01.584845+0000","last_clean":"2024-04-25T16:27:01.584845+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:01.584845+0000","last_undegraded":"2024-04-25T16:27:01.584845+0000","last_fullsized":"2024-04-25T16:27:01.584845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901225+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:02.901225+0000","last_peered":"2024-04-25T16:27:02.901225+0000","last_clean":"2024-04-25T16:27:02.901225+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:02.901225+0000","last_undegraded":"2024-04-25T16:27:02.901225+0000","last_fullsized":"2024-04-25T16:27:02.901225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589204+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:01.589204+0000","last_peered":"2024-04-25T16:27:01.589204+0000","last_clean":"2024-04-25T16:27:01.589204+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:01.589204+0000","last_undegraded":"2024-04-25T16:27:01.589204+0000","last_fullsized":"2024-04-25T16:27:01.589204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900752+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:02.900752+0000","last_peered":"2024-04-25T16:27:02.900752+0000","last_clean":"2024-04-25T16:27:02.900752+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:02.900752+0000","last_undegraded":"2024-04-25T16:27:02.900752+0000","last_fullsized":"2024-04-25T16:27:02.900752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161964+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:02.161964+0000","last_peered":"2024-04-25T16:27:02.161964+0000","last_clean":"2024-04-25T16:27:02.161964+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:02.161964+0000","last_undegraded":"2024-04-25T16:27:02.161964+0000","last_fullsized":"2024-04-25T16:27:02.161964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589641+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:01.589641+0000","last_peered":"2024-04-25T16:27:01.589641+0000","last_clean":"2024-04-25T16:27:01.589641+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:01.589641+0000","last_undegraded":"2024-04-25T16:27:01.589641+0000","last_fullsized":"2024-04-25T16:27:01.589641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900437+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:02.900437+0000","last_peered":"2024-04-25T16:27:02.900437+0000","last_clean":"2024-04-25T16:27:02.900437+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:02.900437+0000","last_undegraded":"2024-04-25T16:27:02.900437+0000","last_fullsized":"2024-04-25T16:27:02.900437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162051+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:02.162051+0000","last_peered":"2024-04-25T16:27:02.162051+0000","last_clean":"2024-04-25T16:27:02.162051+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:02.162051+0000","last_undegraded":"2024-04-25T16:27:02.162051+0000","last_fullsized":"2024-04-25T16:27:02.162051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589525+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:01.589525+0000","last_peered":"2024-04-25T16:27:01.589525+0000","last_clean":"2024-04-25T16:27:01.589525+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:01.589525+0000","last_undegraded":"2024-04-25T16:27:01.589525+0000","last_fullsized":"2024-04-25T16:27:01.589525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591279+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:01.591279+0000","last_peered":"2024-04-25T16:27:01.591279+0000","last_clean":"2024-04-25T16:27:01.591279+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:01.591279+0000","last_undegraded":"2024-04-25T16:27:01.591279+0000","last_fullsized":"2024-04-25T16:27:01.591279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900534+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:02.900534+0000","last_peered":"2024-04-25T16:27:02.900534+0000","last_clean":"2024-04-25T16:27:02.900534+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:02.900534+0000","last_undegraded":"2024-04-25T16:27:02.900534+0000","last_fullsized":"2024-04-25T16:27:02.900534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591334+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:01.591334+0000","last_peered":"2024-04-25T16:27:01.591334+0000","last_clean":"2024-04-25T16:27:01.591334+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:01.591334+0000","last_undegraded":"2024-04-25T16:27:01.591334+0000","last_fullsized":"2024-04-25T16:27:01.591334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584779+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:01.584779+0000","last_peered":"2024-04-25T16:27:01.584779+0000","last_clean":"2024-04-25T16:27:01.584779+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:01.584779+0000","last_undegraded":"2024-04-25T16:27:01.584779+0000","last_fullsized":"2024-04-25T16:27:01.584779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589901+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:01.589901+0000","last_peered":"2024-04-25T16:27:01.589901+0000","last_clean":"2024-04-25T16:27:01.589901+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:01.589901+0000","last_undegraded":"2024-04-25T16:27:01.589901+0000","last_fullsized":"2024-04-25T16:27:01.589901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584240+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:01.584240+0000","last_peered":"2024-04-25T16:27:01.584240+0000","last_clean":"2024-04-25T16:27:01.584240+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:01.584240+0000","last_undegraded":"2024-04-25T16:27:01.584240+0000","last_fullsized":"2024-04-25T16:27:01.584240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900529+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:02.900529+0000","last_peered":"2024-04-25T16:27:02.900529+0000","last_clean":"2024-04-25T16:27:02.900529+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:02.900529+0000","last_undegraded":"2024-04-25T16:27:02.900529+0000","last_fullsized":"2024-04-25T16:27:02.900529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:27:00.605424+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:27:00.604972+0000","last_became_peered":"2024-04-25T16:27:00.604972+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590785+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:01.590785+0000","last_peered":"2024-04-25T16:27:01.590785+0000","last_clean":"2024-04-25T16:27:01.590785+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:01.590785+0000","last_undegraded":"2024-04-25T16:27:01.590785+0000","last_fullsized":"2024-04-25T16:27:01.590785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900076+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:02.900076+0000","last_peered":"2024-04-25T16:27:02.900076+0000","last_clean":"2024-04-25T16:27:02.900076+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:02.900076+0000","last_undegraded":"2024-04-25T16:27:02.900076+0000","last_fullsized":"2024-04-25T16:27:02.900076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591342+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:01.591342+0000","last_peered":"2024-04-25T16:27:01.591342+0000","last_clean":"2024-04-25T16:27:01.591342+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:01.591342+0000","last_undegraded":"2024-04-25T16:27:01.591342+0000","last_fullsized":"2024-04-25T16:27:01.591342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901614+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:02.901614+0000","last_peered":"2024-04-25T16:27:02.901614+0000","last_clean":"2024-04-25T16:27:02.901614+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:02.901614+0000","last_undegraded":"2024-04-25T16:27:02.901614+0000","last_fullsized":"2024-04-25T16:27:02.901614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591218+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:01.591218+0000","last_peered":"2024-04-25T16:27:01.591218+0000","last_clean":"2024-04-25T16:27:01.591218+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:01.591218+0000","last_undegraded":"2024-04-25T16:27:01.591218+0000","last_fullsized":"2024-04-25T16:27:01.591218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590686+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:01.590686+0000","last_peered":"2024-04-25T16:27:01.590686+0000","last_clean":"2024-04-25T16:27:01.590686+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:01.590686+0000","last_undegraded":"2024-04-25T16:27:01.590686+0000","last_fullsized":"2024-04-25T16:27:01.590686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584723+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:01.584723+0000","last_peered":"2024-04-25T16:27:01.584723+0000","last_clean":"2024-04-25T16:27:01.584723+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:01.584723+0000","last_undegraded":"2024-04-25T16:27:01.584723+0000","last_fullsized":"2024-04-25T16:27:01.584723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901145+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:02.901145+0000","last_peered":"2024-04-25T16:27:02.901145+0000","last_clean":"2024-04-25T16:27:02.901145+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:02.901145+0000","last_undegraded":"2024-04-25T16:27:02.901145+0000","last_fullsized":"2024-04-25T16:27:02.901145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.583975+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:01.583975+0000","last_peered":"2024-04-25T16:27:01.583975+0000","last_clean":"2024-04-25T16:27:01.583975+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:01.583975+0000","last_undegraded":"2024-04-25T16:27:01.583975+0000","last_fullsized":"2024-04-25T16:27:01.583975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900898+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:02.900898+0000","last_peered":"2024-04-25T16:27:02.900898+0000","last_clean":"2024-04-25T16:27:02.900898+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:02.900898+0000","last_undegraded":"2024-04-25T16:27:02.900898+0000","last_fullsized":"2024-04-25T16:27:02.900898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900860+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:02.900860+0000","last_peered":"2024-04-25T16:27:02.900860+0000","last_clean":"2024-04-25T16:27:02.900860+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:02.900860+0000","last_undegraded":"2024-04-25T16:27:02.900860+0000","last_fullsized":"2024-04-25T16:27:02.900860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901651+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:02.901651+0000","last_peered":"2024-04-25T16:27:02.901651+0000","last_clean":"2024-04-25T16:27:02.901651+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:02.901651+0000","last_undegraded":"2024-04-25T16:27:02.901651+0000","last_fullsized":"2024-04-25T16:27:02.901651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162233+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:02.162233+0000","last_peered":"2024-04-25T16:27:02.162233+0000","last_clean":"2024-04-25T16:27:02.162233+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:02.162233+0000","last_undegraded":"2024-04-25T16:27:02.162233+0000","last_fullsized":"2024-04-25T16:27:02.162233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901021+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:02.901021+0000","last_peered":"2024-04-25T16:27:02.901021+0000","last_clean":"2024-04-25T16:27:02.901021+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:02.901021+0000","last_undegraded":"2024-04-25T16:27:02.901021+0000","last_fullsized":"2024-04-25T16:27:02.901021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900663+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:02.900663+0000","last_peered":"2024-04-25T16:27:02.900663+0000","last_clean":"2024-04-25T16:27:02.900663+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:02.900663+0000","last_undegraded":"2024-04-25T16:27:02.900663+0000","last_fullsized":"2024-04-25T16:27:02.900663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590164+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:01.590164+0000","last_peered":"2024-04-25T16:27:01.590164+0000","last_clean":"2024-04-25T16:27:01.590164+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:01.590164+0000","last_undegraded":"2024-04-25T16:27:01.590164+0000","last_fullsized":"2024-04-25T16:27:01.590164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162067+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:02.162067+0000","last_peered":"2024-04-25T16:27:02.162067+0000","last_clean":"2024-04-25T16:27:02.162067+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:02.162067+0000","last_undegraded":"2024-04-25T16:27:02.162067+0000","last_fullsized":"2024-04-25T16:27:02.162067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900915+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:02.900915+0000","last_peered":"2024-04-25T16:27:02.900915+0000","last_clean":"2024-04-25T16:27:02.900915+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:02.900915+0000","last_undegraded":"2024-04-25T16:27:02.900915+0000","last_fullsized":"2024-04-25T16:27:02.900915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900165+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:02.900165+0000","last_peered":"2024-04-25T16:27:02.900165+0000","last_clean":"2024-04-25T16:27:02.900165+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:02.900165+0000","last_undegraded":"2024-04-25T16:27:02.900165+0000","last_fullsized":"2024-04-25T16:27:02.900165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900098+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:02.900098+0000","last_peered":"2024-04-25T16:27:02.900098+0000","last_clean":"2024-04-25T16:27:02.900098+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:02.900098+0000","last_undegraded":"2024-04-25T16:27:02.900098+0000","last_fullsized":"2024-04-25T16:27:02.900098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901408+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:02.901408+0000","last_peered":"2024-04-25T16:27:02.901408+0000","last_clean":"2024-04-25T16:27:02.901408+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:02.901408+0000","last_undegraded":"2024-04-25T16:27:02.901408+0000","last_fullsized":"2024-04-25T16:27:02.901408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901450+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:02.901450+0000","last_peered":"2024-04-25T16:27:02.901450+0000","last_clean":"2024-04-25T16:27:02.901450+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:02.901450+0000","last_undegraded":"2024-04-25T16:27:02.901450+0000","last_fullsized":"2024-04-25T16:27:02.901450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900079+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:02.900079+0000","last_peered":"2024-04-25T16:27:02.900079+0000","last_clean":"2024-04-25T16:27:02.900079+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:02.900079+0000","last_undegraded":"2024-04-25T16:27:02.900079+0000","last_fullsized":"2024-04-25T16:27:02.900079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584095+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:01.584095+0000","last_peered":"2024-04-25T16:27:01.584095+0000","last_clean":"2024-04-25T16:27:01.584095+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:01.584095+0000","last_undegraded":"2024-04-25T16:27:01.584095+0000","last_fullsized":"2024-04-25T16:27:01.584095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161817+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:02.161817+0000","last_peered":"2024-04-25T16:27:02.161817+0000","last_clean":"2024-04-25T16:27:02.161817+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:02.161817+0000","last_undegraded":"2024-04-25T16:27:02.161817+0000","last_fullsized":"2024-04-25T16:27:02.161817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900253+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:02.900253+0000","last_peered":"2024-04-25T16:27:02.900253+0000","last_clean":"2024-04-25T16:27:02.900253+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:02.900253+0000","last_undegraded":"2024-04-25T16:27:02.900253+0000","last_fullsized":"2024-04-25T16:27:02.900253+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589943+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:01.589943+0000","last_peered":"2024-04-25T16:27:01.589943+0000","last_clean":"2024-04-25T16:27:01.589943+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:01.589943+0000","last_undegraded":"2024-04-25T16:27:01.589943+0000","last_fullsized":"2024-04-25T16:27:01.589943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900575+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:02.900575+0000","last_peered":"2024-04-25T16:27:02.900575+0000","last_clean":"2024-04-25T16:27:02.900575+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:02.900575+0000","last_undegraded":"2024-04-25T16:27:02.900575+0000","last_fullsized":"2024-04-25T16:27:02.900575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900594+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:02.900594+0000","last_peered":"2024-04-25T16:27:02.900594+0000","last_clean":"2024-04-25T16:27:02.900594+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:02.900594+0000","last_undegraded":"2024-04-25T16:27:02.900594+0000","last_fullsized":"2024-04-25T16:27:02.900594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591187+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:01.591187+0000","last_peered":"2024-04-25T16:27:01.591187+0000","last_clean":"2024-04-25T16:27:01.591187+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:01.591187+0000","last_undegraded":"2024-04-25T16:27:01.591187+0000","last_fullsized":"2024-04-25T16:27:01.591187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161997+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:02.161997+0000","last_peered":"2024-04-25T16:27:02.161997+0000","last_clean":"2024-04-25T16:27:02.161997+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:02.161997+0000","last_undegraded":"2024-04-25T16:27:02.161997+0000","last_fullsized":"2024-04-25T16:27:02.161997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900648+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:02.900648+0000","last_peered":"2024-04-25T16:27:02.900648+0000","last_clean":"2024-04-25T16:27:02.900648+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:02.900648+0000","last_undegraded":"2024-04-25T16:27:02.900648+0000","last_fullsized":"2024-04-25T16:27:02.900648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901955+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:02.901955+0000","last_peered":"2024-04-25T16:27:02.901955+0000","last_clean":"2024-04-25T16:27:02.901955+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:02.901955+0000","last_undegraded":"2024-04-25T16:27:02.901955+0000","last_fullsized":"2024-04-25T16:27:02.901955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900856+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:02.900856+0000","last_peered":"2024-04-25T16:27:02.900856+0000","last_clean":"2024-04-25T16:27:02.900856+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:02.900856+0000","last_undegraded":"2024-04-25T16:27:02.900856+0000","last_fullsized":"2024-04-25T16:27:02.900856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901967+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:02.901967+0000","last_peered":"2024-04-25T16:27:02.901967+0000","last_clean":"2024-04-25T16:27:02.901967+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:02.901967+0000","last_undegraded":"2024-04-25T16:27:02.901967+0000","last_fullsized":"2024-04-25T16:27:02.901967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161498+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:02.161498+0000","last_peered":"2024-04-25T16:27:02.161498+0000","last_clean":"2024-04-25T16:27:02.161498+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:02.161498+0000","last_undegraded":"2024-04-25T16:27:02.161498+0000","last_fullsized":"2024-04-25T16:27:02.161498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900790+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:02.900790+0000","last_peered":"2024-04-25T16:27:02.900790+0000","last_clean":"2024-04-25T16:27:02.900790+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:02.900790+0000","last_undegraded":"2024-04-25T16:27:02.900790+0000","last_fullsized":"2024-04-25T16:27:02.900790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590401+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:01.590401+0000","last_peered":"2024-04-25T16:27:01.590401+0000","last_clean":"2024-04-25T16:27:01.590401+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:01.590401+0000","last_undegraded":"2024-04-25T16:27:01.590401+0000","last_fullsized":"2024-04-25T16:27:01.590401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589995+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:01.589995+0000","last_peered":"2024-04-25T16:27:01.589995+0000","last_clean":"2024-04-25T16:27:01.589995+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:01.589995+0000","last_undegraded":"2024-04-25T16:27:01.589995+0000","last_fullsized":"2024-04-25T16:27:01.589995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900320+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:02.900320+0000","last_peered":"2024-04-25T16:27:02.900320+0000","last_clean":"2024-04-25T16:27:02.900320+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:02.900320+0000","last_undegraded":"2024-04-25T16:27:02.900320+0000","last_fullsized":"2024-04-25T16:27:02.900320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900231+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:02.900231+0000","last_peered":"2024-04-25T16:27:02.900231+0000","last_clean":"2024-04-25T16:27:02.900231+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:02.900231+0000","last_undegraded":"2024-04-25T16:27:02.900231+0000","last_fullsized":"2024-04-25T16:27:02.900231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901358+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:02.901358+0000","last_peered":"2024-04-25T16:27:02.901358+0000","last_clean":"2024-04-25T16:27:02.901358+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:02.901358+0000","last_undegraded":"2024-04-25T16:27:02.901358+0000","last_fullsized":"2024-04-25T16:27:02.901358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589529+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:01.589529+0000","last_peered":"2024-04-25T16:27:01.589529+0000","last_clean":"2024-04-25T16:27:01.589529+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:01.589529+0000","last_undegraded":"2024-04-25T16:27:01.589529+0000","last_fullsized":"2024-04-25T16:27:01.589529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589509+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:01.589509+0000","last_peered":"2024-04-25T16:27:01.589509+0000","last_clean":"2024-04-25T16:27:01.589509+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:01.589509+0000","last_undegraded":"2024-04-25T16:27:01.589509+0000","last_fullsized":"2024-04-25T16:27:01.589509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584625+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:01.584625+0000","last_peered":"2024-04-25T16:27:01.584625+0000","last_clean":"2024-04-25T16:27:01.584625+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:01.584625+0000","last_undegraded":"2024-04-25T16:27:01.584625+0000","last_fullsized":"2024-04-25T16:27:01.584625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900487+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:02.900487+0000","last_peered":"2024-04-25T16:27:02.900487+0000","last_clean":"2024-04-25T16:27:02.900487+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:02.900487+0000","last_undegraded":"2024-04-25T16:27:02.900487+0000","last_fullsized":"2024-04-25T16:27:02.900487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161965+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:02.161965+0000","last_peered":"2024-04-25T16:27:02.161965+0000","last_clean":"2024-04-25T16:27:02.161965+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:02.161965+0000","last_undegraded":"2024-04-25T16:27:02.161965+0000","last_fullsized":"2024-04-25T16:27:02.161965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584525+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:01.584525+0000","last_peered":"2024-04-25T16:27:01.584525+0000","last_clean":"2024-04-25T16:27:01.584525+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:01.584525+0000","last_undegraded":"2024-04-25T16:27:01.584525+0000","last_fullsized":"2024-04-25T16:27:01.584525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902220+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:02.902220+0000","last_peered":"2024-04-25T16:27:02.902220+0000","last_clean":"2024-04-25T16:27:02.902220+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:02.902220+0000","last_undegraded":"2024-04-25T16:27:02.902220+0000","last_fullsized":"2024-04-25T16:27:02.902220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162131+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:02.162131+0000","last_peered":"2024-04-25T16:27:02.162131+0000","last_clean":"2024-04-25T16:27:02.162131+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:02.162131+0000","last_undegraded":"2024-04-25T16:27:02.162131+0000","last_fullsized":"2024-04-25T16:27:02.162131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900492+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:02.900492+0000","last_peered":"2024-04-25T16:27:02.900492+0000","last_clean":"2024-04-25T16:27:02.900492+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:02.900492+0000","last_undegraded":"2024-04-25T16:27:02.900492+0000","last_fullsized":"2024-04-25T16:27:02.900492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:07.253703+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:07.253703+0000","last_peered":"2024-04-25T16:27:07.253703+0000","last_clean":"2024-04-25T16:27:07.253703+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:07.253703+0000","last_undegraded":"2024-04-25T16:27:07.253703+0000","last_fullsized":"2024-04-25T16:27:07.253703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900859+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:02.900859+0000","last_peered":"2024-04-25T16:27:02.900859+0000","last_clean":"2024-04-25T16:27:02.900859+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:02.900859+0000","last_undegraded":"2024-04-25T16:27:02.900859+0000","last_fullsized":"2024-04-25T16:27:02.900859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590243+0000","last_change":"2024-04-25T16:26:56.570117+0000","last_active":"2024-04-25T16:27:01.590243+0000","last_peered":"2024-04-25T16:27:01.590243+0000","last_clean":"2024-04-25T16:27:01.590243+0000","last_became_active":"2024-04-25T16:26:56.569817+0000","last_became_peered":"2024-04-25T16:26:56.569817+0000","last_unstale":"2024-04-25T16:27:01.590243+0000","last_undegraded":"2024-04-25T16:27:01.590243+0000","last_fullsized":"2024-04-25T16:27:01.590243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:12.099635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900579+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:02.900579+0000","last_peered":"2024-04-25T16:27:02.900579+0000","last_clean":"2024-04-25T16:27:02.900579+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:02.900579+0000","last_undegraded":"2024-04-25T16:27:02.900579+0000","last_fullsized":"2024-04-25T16:27:02.900579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589933+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:01.589933+0000","last_peered":"2024-04-25T16:27:01.589933+0000","last_clean":"2024-04-25T16:27:01.589933+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:01.589933+0000","last_undegraded":"2024-04-25T16:27:01.589933+0000","last_fullsized":"2024-04-25T16:27:01.589933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162032+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:02.162032+0000","last_peered":"2024-04-25T16:27:02.162032+0000","last_clean":"2024-04-25T16:27:02.162032+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:02.162032+0000","last_undegraded":"2024-04-25T16:27:02.162032+0000","last_fullsized":"2024-04-25T16:27:02.162032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584281+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:01.584281+0000","last_peered":"2024-04-25T16:27:01.584281+0000","last_clean":"2024-04-25T16:27:01.584281+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:01.584281+0000","last_undegraded":"2024-04-25T16:27:01.584281+0000","last_fullsized":"2024-04-25T16:27:01.584281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161984+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:02.161984+0000","last_peered":"2024-04-25T16:27:02.161984+0000","last_clean":"2024-04-25T16:27:02.161984+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:02.161984+0000","last_undegraded":"2024-04-25T16:27:02.161984+0000","last_fullsized":"2024-04-25T16:27:02.161984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591106+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:01.591106+0000","last_peered":"2024-04-25T16:27:01.591106+0000","last_clean":"2024-04-25T16:27:01.591106+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:01.591106+0000","last_undegraded":"2024-04-25T16:27:01.591106+0000","last_fullsized":"2024-04-25T16:27:01.591106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584144+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:01.584144+0000","last_peered":"2024-04-25T16:27:01.584144+0000","last_clean":"2024-04-25T16:27:01.584144+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:01.584144+0000","last_undegraded":"2024-04-25T16:27:01.584144+0000","last_fullsized":"2024-04-25T16:27:01.584144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901211+0000","last_change":"2024-04-25T16:27:00.605248+0000","last_active":"2024-04-25T16:27:02.901211+0000","last_peered":"2024-04-25T16:27:02.901211+0000","last_clean":"2024-04-25T16:27:02.901211+0000","last_became_active":"2024-04-25T16:27:00.604746+0000","last_became_peered":"2024-04-25T16:27:00.604746+0000","last_unstale":"2024-04-25T16:27:02.901211+0000","last_undegraded":"2024-04-25T16:27:02.901211+0000","last_fullsized":"2024-04-25T16:27:02.901211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900313+0000","last_change":"2024-04-25T16:26:56.553599+0000","last_active":"2024-04-25T16:27:02.900313+0000","last_peered":"2024-04-25T16:27:02.900313+0000","last_clean":"2024-04-25T16:27:02.900313+0000","last_became_active":"2024-04-25T16:26:56.553304+0000","last_became_peered":"2024-04-25T16:26:56.553304+0000","last_unstale":"2024-04-25T16:27:02.900313+0000","last_undegraded":"2024-04-25T16:27:02.900313+0000","last_fullsized":"2024-04-25T16:27:02.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161615+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:02.161615+0000","last_peered":"2024-04-25T16:27:02.161615+0000","last_clean":"2024-04-25T16:27:02.161615+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:02.161615+0000","last_undegraded":"2024-04-25T16:27:02.161615+0000","last_fullsized":"2024-04-25T16:27:02.161615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591033+0000","last_change":"2024-04-25T16:27:00.587907+0000","last_active":"2024-04-25T16:27:01.591033+0000","last_peered":"2024-04-25T16:27:01.591033+0000","last_clean":"2024-04-25T16:27:01.591033+0000","last_became_active":"2024-04-25T16:27:00.587281+0000","last_became_peered":"2024-04-25T16:27:00.587281+0000","last_unstale":"2024-04-25T16:27:01.591033+0000","last_undegraded":"2024-04-25T16:27:01.591033+0000","last_fullsized":"2024-04-25T16:27:01.591033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584123+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:01.584123+0000","last_peered":"2024-04-25T16:27:01.584123+0000","last_clean":"2024-04-25T16:27:01.584123+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:01.584123+0000","last_undegraded":"2024-04-25T16:27:01.584123+0000","last_fullsized":"2024-04-25T16:27:01.584123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584701+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:01.584701+0000","last_peered":"2024-04-25T16:27:01.584701+0000","last_clean":"2024-04-25T16:27:01.584701+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:01.584701+0000","last_undegraded":"2024-04-25T16:27:01.584701+0000","last_fullsized":"2024-04-25T16:27:01.584701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591433+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:01.591433+0000","last_peered":"2024-04-25T16:27:01.591433+0000","last_clean":"2024-04-25T16:27:01.591433+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:01.591433+0000","last_undegraded":"2024-04-25T16:27:01.591433+0000","last_fullsized":"2024-04-25T16:27:01.591433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902190+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:02.902190+0000","last_peered":"2024-04-25T16:27:02.902190+0000","last_clean":"2024-04-25T16:27:02.902190+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:02.902190+0000","last_undegraded":"2024-04-25T16:27:02.902190+0000","last_fullsized":"2024-04-25T16:27:02.902190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900327+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:02.900327+0000","last_peered":"2024-04-25T16:27:02.900327+0000","last_clean":"2024-04-25T16:27:02.900327+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:02.900327+0000","last_undegraded":"2024-04-25T16:27:02.900327+0000","last_fullsized":"2024-04-25T16:27:02.900327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584664+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:01.584664+0000","last_peered":"2024-04-25T16:27:01.584664+0000","last_clean":"2024-04-25T16:27:01.584664+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:01.584664+0000","last_undegraded":"2024-04-25T16:27:01.584664+0000","last_fullsized":"2024-04-25T16:27:01.584664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589742+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:01.589742+0000","last_peered":"2024-04-25T16:27:01.589742+0000","last_clean":"2024-04-25T16:27:01.589742+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:01.589742+0000","last_undegraded":"2024-04-25T16:27:01.589742+0000","last_fullsized":"2024-04-25T16:27:01.589742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584547+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:01.584547+0000","last_peered":"2024-04-25T16:27:01.584547+0000","last_clean":"2024-04-25T16:27:01.584547+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:01.584547+0000","last_undegraded":"2024-04-25T16:27:01.584547+0000","last_fullsized":"2024-04-25T16:27:01.584547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589145+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:01.589145+0000","last_peered":"2024-04-25T16:27:01.589145+0000","last_clean":"2024-04-25T16:27:01.589145+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:01.589145+0000","last_undegraded":"2024-04-25T16:27:01.589145+0000","last_fullsized":"2024-04-25T16:27:01.589145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.899980+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:02.899980+0000","last_peered":"2024-04-25T16:27:02.899980+0000","last_clean":"2024-04-25T16:27:02.899980+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:02.899980+0000","last_undegraded":"2024-04-25T16:27:02.899980+0000","last_fullsized":"2024-04-25T16:27:02.899980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161855+0000","last_change":"2024-04-25T16:26:56.558625+0000","last_active":"2024-04-25T16:27:02.161855+0000","last_peered":"2024-04-25T16:27:02.161855+0000","last_clean":"2024-04-25T16:27:02.161855+0000","last_became_active":"2024-04-25T16:26:56.558322+0000","last_became_peered":"2024-04-25T16:26:56.558322+0000","last_unstale":"2024-04-25T16:27:02.161855+0000","last_undegraded":"2024-04-25T16:27:02.161855+0000","last_fullsized":"2024-04-25T16:27:02.161855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902073+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:02.902073+0000","last_peered":"2024-04-25T16:27:02.902073+0000","last_clean":"2024-04-25T16:27:02.902073+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:02.902073+0000","last_undegraded":"2024-04-25T16:27:02.902073+0000","last_fullsized":"2024-04-25T16:27:02.902073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584798+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:01.584798+0000","last_peered":"2024-04-25T16:27:01.584798+0000","last_clean":"2024-04-25T16:27:01.584798+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:01.584798+0000","last_undegraded":"2024-04-25T16:27:01.584798+0000","last_fullsized":"2024-04-25T16:27:01.584798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590378+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:01.590378+0000","last_peered":"2024-04-25T16:27:01.590378+0000","last_clean":"2024-04-25T16:27:01.590378+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:01.590378+0000","last_undegraded":"2024-04-25T16:27:01.590378+0000","last_fullsized":"2024-04-25T16:27:01.590378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584670+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:01.584670+0000","last_peered":"2024-04-25T16:27:01.584670+0000","last_clean":"2024-04-25T16:27:01.584670+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:01.584670+0000","last_undegraded":"2024-04-25T16:27:01.584670+0000","last_fullsized":"2024-04-25T16:27:01.584670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161594+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:02.161594+0000","last_peered":"2024-04-25T16:27:02.161594+0000","last_clean":"2024-04-25T16:27:02.161594+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:02.161594+0000","last_undegraded":"2024-04-25T16:27:02.161594+0000","last_fullsized":"2024-04-25T16:27:02.161594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584586+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:01.584586+0000","last_peered":"2024-04-25T16:27:01.584586+0000","last_clean":"2024-04-25T16:27:01.584586+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:01.584586+0000","last_undegraded":"2024-04-25T16:27:01.584586+0000","last_fullsized":"2024-04-25T16:27:01.584586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161810+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:02.161810+0000","last_peered":"2024-04-25T16:27:02.161810+0000","last_clean":"2024-04-25T16:27:02.161810+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:02.161810+0000","last_undegraded":"2024-04-25T16:27:02.161810+0000","last_fullsized":"2024-04-25T16:27:02.161810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901772+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:02.901772+0000","last_peered":"2024-04-25T16:27:02.901772+0000","last_clean":"2024-04-25T16:27:02.901772+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:02.901772+0000","last_undegraded":"2024-04-25T16:27:02.901772+0000","last_fullsized":"2024-04-25T16:27:02.901772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589540+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:01.589540+0000","last_peered":"2024-04-25T16:27:01.589540+0000","last_clean":"2024-04-25T16:27:01.589540+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:01.589540+0000","last_undegraded":"2024-04-25T16:27:01.589540+0000","last_fullsized":"2024-04-25T16:27:01.589540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900759+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:02.900759+0000","last_peered":"2024-04-25T16:27:02.900759+0000","last_clean":"2024-04-25T16:27:02.900759+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:02.900759+0000","last_undegraded":"2024-04-25T16:27:02.900759+0000","last_fullsized":"2024-04-25T16:27:02.900759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901860+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:02.901860+0000","last_peered":"2024-04-25T16:27:02.901860+0000","last_clean":"2024-04-25T16:27:02.901860+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:02.901860+0000","last_undegraded":"2024-04-25T16:27:02.901860+0000","last_fullsized":"2024-04-25T16:27:02.901860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162241+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:02.162241+0000","last_peered":"2024-04-25T16:27:02.162241+0000","last_clean":"2024-04-25T16:27:02.162241+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:02.162241+0000","last_undegraded":"2024-04-25T16:27:02.162241+0000","last_fullsized":"2024-04-25T16:27:02.162241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161412+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:02.161412+0000","last_peered":"2024-04-25T16:27:02.161412+0000","last_clean":"2024-04-25T16:27:02.161412+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:02.161412+0000","last_undegraded":"2024-04-25T16:27:02.161412+0000","last_fullsized":"2024-04-25T16:27:02.161412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900423+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:02.900423+0000","last_peered":"2024-04-25T16:27:02.900423+0000","last_clean":"2024-04-25T16:27:02.900423+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:02.900423+0000","last_undegraded":"2024-04-25T16:27:02.900423+0000","last_fullsized":"2024-04-25T16:27:02.900423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584351+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:01.584351+0000","last_peered":"2024-04-25T16:27:01.584351+0000","last_clean":"2024-04-25T16:27:01.584351+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:01.584351+0000","last_undegraded":"2024-04-25T16:27:01.584351+0000","last_fullsized":"2024-04-25T16:27:01.584351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590124+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:01.590124+0000","last_peered":"2024-04-25T16:27:01.590124+0000","last_clean":"2024-04-25T16:27:01.590124+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:01.590124+0000","last_undegraded":"2024-04-25T16:27:01.590124+0000","last_fullsized":"2024-04-25T16:27:01.590124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590011+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:01.590011+0000","last_peered":"2024-04-25T16:27:01.590011+0000","last_clean":"2024-04-25T16:27:01.590011+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:01.590011+0000","last_undegraded":"2024-04-25T16:27:01.590011+0000","last_fullsized":"2024-04-25T16:27:01.590011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590256+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:01.590256+0000","last_peered":"2024-04-25T16:27:01.590256+0000","last_clean":"2024-04-25T16:27:01.590256+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:01.590256+0000","last_undegraded":"2024-04-25T16:27:01.590256+0000","last_fullsized":"2024-04-25T16:27:01.590256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900847+0000","last_change":"2024-04-25T16:26:56.572739+0000","last_active":"2024-04-25T16:27:02.900847+0000","last_peered":"2024-04-25T16:27:02.900847+0000","last_clean":"2024-04-25T16:27:02.900847+0000","last_became_active":"2024-04-25T16:26:56.571251+0000","last_became_peered":"2024-04-25T16:26:56.571251+0000","last_unstale":"2024-04-25T16:27:02.900847+0000","last_undegraded":"2024-04-25T16:27:02.900847+0000","last_fullsized":"2024-04-25T16:27:02.900847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584005+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:01.584005+0000","last_peered":"2024-04-25T16:27:01.584005+0000","last_clean":"2024-04-25T16:27:01.584005+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:01.584005+0000","last_undegraded":"2024-04-25T16:27:01.584005+0000","last_fullsized":"2024-04-25T16:27:01.584005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900188+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:02.900188+0000","last_peered":"2024-04-25T16:27:02.900188+0000","last_clean":"2024-04-25T16:27:02.900188+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:02.900188+0000","last_undegraded":"2024-04-25T16:27:02.900188+0000","last_fullsized":"2024-04-25T16:27:02.900188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161603+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:02.161603+0000","last_peered":"2024-04-25T16:27:02.161603+0000","last_clean":"2024-04-25T16:27:02.161603+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:02.161603+0000","last_undegraded":"2024-04-25T16:27:02.161603+0000","last_fullsized":"2024-04-25T16:27:02.161603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161799+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:02.161799+0000","last_peered":"2024-04-25T16:27:02.161799+0000","last_clean":"2024-04-25T16:27:02.161799+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:02.161799+0000","last_undegraded":"2024-04-25T16:27:02.161799+0000","last_fullsized":"2024-04-25T16:27:02.161799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900832+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:02.900832+0000","last_peered":"2024-04-25T16:27:02.900832+0000","last_clean":"2024-04-25T16:27:02.900832+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:02.900832+0000","last_undegraded":"2024-04-25T16:27:02.900832+0000","last_fullsized":"2024-04-25T16:27:02.900832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901018+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:02.901018+0000","last_peered":"2024-04-25T16:27:02.901018+0000","last_clean":"2024-04-25T16:27:02.901018+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:02.901018+0000","last_undegraded":"2024-04-25T16:27:02.901018+0000","last_fullsized":"2024-04-25T16:27:02.901018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900715+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:02.900715+0000","last_peered":"2024-04-25T16:27:02.900715+0000","last_clean":"2024-04-25T16:27:02.900715+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:02.900715+0000","last_undegraded":"2024-04-25T16:27:02.900715+0000","last_fullsized":"2024-04-25T16:27:02.900715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":103933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":694813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:20.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:20.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.778+0000 7f1fbe025640 1 Processor -- start 2024-04-25T16:27:20.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.778+0000 7f1fbe025640 1 -- start start 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fbe025640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1fb80b5230 0x7f1fb8160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fbe025640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb80b5a30 con 0x7f1fb8164210 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fbe025640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb80d5610 con 0x7f1fb8164d30 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fbe025640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb806ca10 con 0x7f1fb80b5230 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb6ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1fb8164210 0x7f1fb8164620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42046/0 (socket says 172.21.15.94:42046) 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/3488400513 learned_addr learned my addr 172.21.15.94:0/3488400513 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/3488400513 >> v2:172.21.15.138:3300/0 conn(0x7f1fb80b5230 0x7f1fb8160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb67fc640 1 -- 172.21.15.94:0/3488400513 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1048743081 0 0) 0x7f1fac001110 con 0x7f1fb8164210 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb67fc640 1 -- 172.21.15.94:0/3488400513 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa0003520 con 0x7f1fb8164210 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb77fe640 1 -- 172.21.15.94:0/3488400513 >> v1:172.21.15.138:6789/0 conn(0x7f1fb8164d30 legacy=0x7f1fb806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb77fe640 1 -- 172.21.15.94:0/3488400513 >> v1:172.21.15.94:6789/0 conn(0x7f1fb8164210 legacy=0x7f1fb8164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb67fc640 1 -- 172.21.15.94:0/3488400513 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3919091113 0 0) 0x7f1fa0003520 con 0x7f1fb8164210 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.780+0000 7f1fb77fe640 1 -- 172.21.15.94:0/3488400513 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb80d58f0 con 0x7f1fb80b5230 2024-04-25T16:27:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.781+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/3488400513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8160790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80097a0 tx=0x7f1fa802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777872262b3c756c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.781+0000 7f1fb67fc640 1 -- 172.21.15.94:0/3488400513 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8037070 con 0x7f1fb80b5230 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.781+0000 7f1fb67fc640 1 -- 172.21.15.94:0/3488400513 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa802abc0 con 0x7f1fb80b5230 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.781+0000 7f1fb67fc640 1 -- 172.21.15.94:0/3488400513 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac001840 con 0x7f1fb80b5230 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 -- 172.21.15.94:0/3488400513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 msgr2=0x7f1fb8160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 --2- 172.21.15.94:0/3488400513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8160790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80097a0 tx=0x7f1fa802a130 comp rx=0 tx=0).stop 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 -- 172.21.15.94:0/3488400513 shutdown_connections 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 --2- 172.21.15.94:0/3488400513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8160790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 -- 172.21.15.94:0/3488400513 >> 172.21.15.94:0/3488400513 conn(0x7f1fb80d3570 msgr2=0x7f1fb80b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 -- 172.21.15.94:0/3488400513 shutdown_connections 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.782+0000 7f1fbe025640 1 -- 172.21.15.94:0/3488400513 wait complete. 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.784+0000 7f1fbe025640 1 Processor -- start 2024-04-25T16:27:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.784+0000 7f1fbe025640 1 -- start start 2024-04-25T16:27:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.785+0000 7f1fbe025640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.785+0000 7f1fbe025640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8164210 0x7f1fb815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.785+0000 7f1fbe025640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 0x7f1fb81fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.785+0000 7f1fbe025640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb80b9870 con 0x7f1fb8164d30 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.785+0000 7f1fbe025640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb80b96f0 con 0x7f1fb80b5230 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.785+0000 7f1fbe025640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb806d5b0 con 0x7f1fb8164210 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 0x7f1fb81fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 0x7f1fb81fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35838/0 (socket says 172.21.15.94:35838) 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 -- 172.21.15.94:0/1503028515 learned_addr learned my addr 172.21.15.94:0/1503028515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8164210 0x7f1fb815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 -- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8164210 msgr2=0x7f1fb815b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8164210 0x7f1fb815b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 -- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 msgr2=0x7f1fb8161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 -- 172.21.15.94:0/1503028515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb8160d90 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 0x7f1fb81fa190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c6e0 tx=0x7f1fb000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0003b70 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb0003d30 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0005310 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb8161070 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.786+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb81faf30 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.787+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb806a180 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.788+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1fb00054b0 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.788+0000 7f1f9ffff640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1fa40744b0 0x7f1fa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.788+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1fa40744b0 0x7f1fa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.789+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1fb00953d0 con 0x7f1fb8164d30 2024-04-25T16:27:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.789+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1fa40744b0 0x7f1fa40769c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8034000 tx=0x7f1fa802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.793+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fb005f820 con 0x7f1fb8164d30 2024-04-25T16:27:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.962+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fb80bb590 con 0x7f1fa40744b0 2024-04-25T16:27:20.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.970+0000 7f1f9ffff640 1 -- 172.21.15.94:0/1503028515 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f1fb80bb590 con 0x7f1fa40744b0 2024-04-25T16:27:20.975 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:20.976 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:20.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.973+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1fa40744b0 msgr2=0x7f1fa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.973+0000 7f1fbe025640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1fa40744b0 0x7f1fa40769c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8034000 tx=0x7f1fa802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:20.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.973+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 msgr2=0x7f1fb81fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.973+0000 7f1fbe025640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 0x7f1fb81fa190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c6e0 tx=0x7f1fb000cba0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 shutdown_connections 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1fa40744b0 0x7f1fa40769c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb8164d30 0x7f1fb81fa190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8164210 0x7f1fb815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 --2- 172.21.15.94:0/1503028515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb80b5230 0x7f1fb8161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 >> 172.21.15.94:0/1503028515 conn(0x7f1fb80d3570 msgr2=0x7f1fb806d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 shutdown_connections 2024-04-25T16:27:20.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:20.974+0000 7f1fbe025640 1 -- 172.21.15.94:0/1503028515 wait complete. 2024-04-25T16:27:20.986 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T16:27:19.302432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236072,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738648,"statfs":{"total":773094113280,"available":772852375552,"internally_reserved":0,"allocated":3350528,"data_stored":2020472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003159"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901527+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:02.901527+0000","last_peered":"2024-04-25T16:27:02.901527+0000","last_clean":"2024-04-25T16:27:02.901527+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:02.901527+0000","last_undegraded":"2024-04-25T16:27:02.901527+0000","last_fullsized":"2024-04-25T16:27:02.901527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584607+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:01.584607+0000","last_peered":"2024-04-25T16:27:01.584607+0000","last_clean":"2024-04-25T16:27:01.584607+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:01.584607+0000","last_undegraded":"2024-04-25T16:27:01.584607+0000","last_fullsized":"2024-04-25T16:27:01.584607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584101+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:01.584101+0000","last_peered":"2024-04-25T16:27:01.584101+0000","last_clean":"2024-04-25T16:27:01.584101+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:01.584101+0000","last_undegraded":"2024-04-25T16:27:01.584101+0000","last_fullsized":"2024-04-25T16:27:01.584101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584845+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:01.584845+0000","last_peered":"2024-04-25T16:27:01.584845+0000","last_clean":"2024-04-25T16:27:01.584845+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:01.584845+0000","last_undegraded":"2024-04-25T16:27:01.584845+0000","last_fullsized":"2024-04-25T16:27:01.584845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901225+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:02.901225+0000","last_peered":"2024-04-25T16:27:02.901225+0000","last_clean":"2024-04-25T16:27:02.901225+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:02.901225+0000","last_undegraded":"2024-04-25T16:27:02.901225+0000","last_fullsized":"2024-04-25T16:27:02.901225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589204+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:01.589204+0000","last_peered":"2024-04-25T16:27:01.589204+0000","last_clean":"2024-04-25T16:27:01.589204+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:01.589204+0000","last_undegraded":"2024-04-25T16:27:01.589204+0000","last_fullsized":"2024-04-25T16:27:01.589204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900752+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:02.900752+0000","last_peered":"2024-04-25T16:27:02.900752+0000","last_clean":"2024-04-25T16:27:02.900752+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:02.900752+0000","last_undegraded":"2024-04-25T16:27:02.900752+0000","last_fullsized":"2024-04-25T16:27:02.900752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161964+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:02.161964+0000","last_peered":"2024-04-25T16:27:02.161964+0000","last_clean":"2024-04-25T16:27:02.161964+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:02.161964+0000","last_undegraded":"2024-04-25T16:27:02.161964+0000","last_fullsized":"2024-04-25T16:27:02.161964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589641+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:01.589641+0000","last_peered":"2024-04-25T16:27:01.589641+0000","last_clean":"2024-04-25T16:27:01.589641+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:01.589641+0000","last_undegraded":"2024-04-25T16:27:01.589641+0000","last_fullsized":"2024-04-25T16:27:01.589641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900437+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:02.900437+0000","last_peered":"2024-04-25T16:27:02.900437+0000","last_clean":"2024-04-25T16:27:02.900437+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:02.900437+0000","last_undegraded":"2024-04-25T16:27:02.900437+0000","last_fullsized":"2024-04-25T16:27:02.900437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162051+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:02.162051+0000","last_peered":"2024-04-25T16:27:02.162051+0000","last_clean":"2024-04-25T16:27:02.162051+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:02.162051+0000","last_undegraded":"2024-04-25T16:27:02.162051+0000","last_fullsized":"2024-04-25T16:27:02.162051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589525+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:01.589525+0000","last_peered":"2024-04-25T16:27:01.589525+0000","last_clean":"2024-04-25T16:27:01.589525+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:01.589525+0000","last_undegraded":"2024-04-25T16:27:01.589525+0000","last_fullsized":"2024-04-25T16:27:01.589525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591279+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:01.591279+0000","last_peered":"2024-04-25T16:27:01.591279+0000","last_clean":"2024-04-25T16:27:01.591279+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:01.591279+0000","last_undegraded":"2024-04-25T16:27:01.591279+0000","last_fullsized":"2024-04-25T16:27:01.591279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900534+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:02.900534+0000","last_peered":"2024-04-25T16:27:02.900534+0000","last_clean":"2024-04-25T16:27:02.900534+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:02.900534+0000","last_undegraded":"2024-04-25T16:27:02.900534+0000","last_fullsized":"2024-04-25T16:27:02.900534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591334+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:01.591334+0000","last_peered":"2024-04-25T16:27:01.591334+0000","last_clean":"2024-04-25T16:27:01.591334+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:01.591334+0000","last_undegraded":"2024-04-25T16:27:01.591334+0000","last_fullsized":"2024-04-25T16:27:01.591334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584779+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:01.584779+0000","last_peered":"2024-04-25T16:27:01.584779+0000","last_clean":"2024-04-25T16:27:01.584779+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:01.584779+0000","last_undegraded":"2024-04-25T16:27:01.584779+0000","last_fullsized":"2024-04-25T16:27:01.584779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589901+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:01.589901+0000","last_peered":"2024-04-25T16:27:01.589901+0000","last_clean":"2024-04-25T16:27:01.589901+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:01.589901+0000","last_undegraded":"2024-04-25T16:27:01.589901+0000","last_fullsized":"2024-04-25T16:27:01.589901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584240+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:01.584240+0000","last_peered":"2024-04-25T16:27:01.584240+0000","last_clean":"2024-04-25T16:27:01.584240+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:01.584240+0000","last_undegraded":"2024-04-25T16:27:01.584240+0000","last_fullsized":"2024-04-25T16:27:01.584240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900529+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:02.900529+0000","last_peered":"2024-04-25T16:27:02.900529+0000","last_clean":"2024-04-25T16:27:02.900529+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:02.900529+0000","last_undegraded":"2024-04-25T16:27:02.900529+0000","last_fullsized":"2024-04-25T16:27:02.900529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:27:00.605424+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:27:00.604972+0000","last_became_peered":"2024-04-25T16:27:00.604972+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590785+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:01.590785+0000","last_peered":"2024-04-25T16:27:01.590785+0000","last_clean":"2024-04-25T16:27:01.590785+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:01.590785+0000","last_undegraded":"2024-04-25T16:27:01.590785+0000","last_fullsized":"2024-04-25T16:27:01.590785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900076+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:02.900076+0000","last_peered":"2024-04-25T16:27:02.900076+0000","last_clean":"2024-04-25T16:27:02.900076+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:02.900076+0000","last_undegraded":"2024-04-25T16:27:02.900076+0000","last_fullsized":"2024-04-25T16:27:02.900076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591342+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:01.591342+0000","last_peered":"2024-04-25T16:27:01.591342+0000","last_clean":"2024-04-25T16:27:01.591342+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:01.591342+0000","last_undegraded":"2024-04-25T16:27:01.591342+0000","last_fullsized":"2024-04-25T16:27:01.591342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901614+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:02.901614+0000","last_peered":"2024-04-25T16:27:02.901614+0000","last_clean":"2024-04-25T16:27:02.901614+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:02.901614+0000","last_undegraded":"2024-04-25T16:27:02.901614+0000","last_fullsized":"2024-04-25T16:27:02.901614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591218+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:01.591218+0000","last_peered":"2024-04-25T16:27:01.591218+0000","last_clean":"2024-04-25T16:27:01.591218+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:01.591218+0000","last_undegraded":"2024-04-25T16:27:01.591218+0000","last_fullsized":"2024-04-25T16:27:01.591218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590686+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:01.590686+0000","last_peered":"2024-04-25T16:27:01.590686+0000","last_clean":"2024-04-25T16:27:01.590686+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:01.590686+0000","last_undegraded":"2024-04-25T16:27:01.590686+0000","last_fullsized":"2024-04-25T16:27:01.590686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584723+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:01.584723+0000","last_peered":"2024-04-25T16:27:01.584723+0000","last_clean":"2024-04-25T16:27:01.584723+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:01.584723+0000","last_undegraded":"2024-04-25T16:27:01.584723+0000","last_fullsized":"2024-04-25T16:27:01.584723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901145+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:02.901145+0000","last_peered":"2024-04-25T16:27:02.901145+0000","last_clean":"2024-04-25T16:27:02.901145+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:02.901145+0000","last_undegraded":"2024-04-25T16:27:02.901145+0000","last_fullsized":"2024-04-25T16:27:02.901145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.583975+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:01.583975+0000","last_peered":"2024-04-25T16:27:01.583975+0000","last_clean":"2024-04-25T16:27:01.583975+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:01.583975+0000","last_undegraded":"2024-04-25T16:27:01.583975+0000","last_fullsized":"2024-04-25T16:27:01.583975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900898+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:02.900898+0000","last_peered":"2024-04-25T16:27:02.900898+0000","last_clean":"2024-04-25T16:27:02.900898+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:02.900898+0000","last_undegraded":"2024-04-25T16:27:02.900898+0000","last_fullsized":"2024-04-25T16:27:02.900898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900860+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:02.900860+0000","last_peered":"2024-04-25T16:27:02.900860+0000","last_clean":"2024-04-25T16:27:02.900860+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:02.900860+0000","last_undegraded":"2024-04-25T16:27:02.900860+0000","last_fullsized":"2024-04-25T16:27:02.900860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901651+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:02.901651+0000","last_peered":"2024-04-25T16:27:02.901651+0000","last_clean":"2024-04-25T16:27:02.901651+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:02.901651+0000","last_undegraded":"2024-04-25T16:27:02.901651+0000","last_fullsized":"2024-04-25T16:27:02.901651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162233+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:02.162233+0000","last_peered":"2024-04-25T16:27:02.162233+0000","last_clean":"2024-04-25T16:27:02.162233+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:02.162233+0000","last_undegraded":"2024-04-25T16:27:02.162233+0000","last_fullsized":"2024-04-25T16:27:02.162233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901021+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:02.901021+0000","last_peered":"2024-04-25T16:27:02.901021+0000","last_clean":"2024-04-25T16:27:02.901021+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:02.901021+0000","last_undegraded":"2024-04-25T16:27:02.901021+0000","last_fullsized":"2024-04-25T16:27:02.901021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900663+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:02.900663+0000","last_peered":"2024-04-25T16:27:02.900663+0000","last_clean":"2024-04-25T16:27:02.900663+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:02.900663+0000","last_undegraded":"2024-04-25T16:27:02.900663+0000","last_fullsized":"2024-04-25T16:27:02.900663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590164+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:01.590164+0000","last_peered":"2024-04-25T16:27:01.590164+0000","last_clean":"2024-04-25T16:27:01.590164+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:01.590164+0000","last_undegraded":"2024-04-25T16:27:01.590164+0000","last_fullsized":"2024-04-25T16:27:01.590164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162067+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:02.162067+0000","last_peered":"2024-04-25T16:27:02.162067+0000","last_clean":"2024-04-25T16:27:02.162067+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:02.162067+0000","last_undegraded":"2024-04-25T16:27:02.162067+0000","last_fullsized":"2024-04-25T16:27:02.162067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900915+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:02.900915+0000","last_peered":"2024-04-25T16:27:02.900915+0000","last_clean":"2024-04-25T16:27:02.900915+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:02.900915+0000","last_undegraded":"2024-04-25T16:27:02.900915+0000","last_fullsized":"2024-04-25T16:27:02.900915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900165+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:02.900165+0000","last_peered":"2024-04-25T16:27:02.900165+0000","last_clean":"2024-04-25T16:27:02.900165+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:02.900165+0000","last_undegraded":"2024-04-25T16:27:02.900165+0000","last_fullsized":"2024-04-25T16:27:02.900165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900098+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:02.900098+0000","last_peered":"2024-04-25T16:27:02.900098+0000","last_clean":"2024-04-25T16:27:02.900098+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:02.900098+0000","last_undegraded":"2024-04-25T16:27:02.900098+0000","last_fullsized":"2024-04-25T16:27:02.900098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901408+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:02.901408+0000","last_peered":"2024-04-25T16:27:02.901408+0000","last_clean":"2024-04-25T16:27:02.901408+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:02.901408+0000","last_undegraded":"2024-04-25T16:27:02.901408+0000","last_fullsized":"2024-04-25T16:27:02.901408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901450+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:02.901450+0000","last_peered":"2024-04-25T16:27:02.901450+0000","last_clean":"2024-04-25T16:27:02.901450+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:02.901450+0000","last_undegraded":"2024-04-25T16:27:02.901450+0000","last_fullsized":"2024-04-25T16:27:02.901450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900079+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:02.900079+0000","last_peered":"2024-04-25T16:27:02.900079+0000","last_clean":"2024-04-25T16:27:02.900079+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:02.900079+0000","last_undegraded":"2024-04-25T16:27:02.900079+0000","last_fullsized":"2024-04-25T16:27:02.900079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584095+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:01.584095+0000","last_peered":"2024-04-25T16:27:01.584095+0000","last_clean":"2024-04-25T16:27:01.584095+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:01.584095+0000","last_undegraded":"2024-04-25T16:27:01.584095+0000","last_fullsized":"2024-04-25T16:27:01.584095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161817+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:02.161817+0000","last_peered":"2024-04-25T16:27:02.161817+0000","last_clean":"2024-04-25T16:27:02.161817+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:02.161817+0000","last_undegraded":"2024-04-25T16:27:02.161817+0000","last_fullsized":"2024-04-25T16:27:02.161817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900253+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:02.900253+0000","last_peered":"2024-04-25T16:27:02.900253+0000","last_clean":"2024-04-25T16:27:02.900253+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:02.900253+0000","last_undegraded":"2024-04-25T16:27:02.900253+0000","last_fullsized":"2024-04-25T16:27:02.900253+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589943+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:01.589943+0000","last_peered":"2024-04-25T16:27:01.589943+0000","last_clean":"2024-04-25T16:27:01.589943+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:01.589943+0000","last_undegraded":"2024-04-25T16:27:01.589943+0000","last_fullsized":"2024-04-25T16:27:01.589943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900575+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:02.900575+0000","last_peered":"2024-04-25T16:27:02.900575+0000","last_clean":"2024-04-25T16:27:02.900575+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:02.900575+0000","last_undegraded":"2024-04-25T16:27:02.900575+0000","last_fullsized":"2024-04-25T16:27:02.900575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900594+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:02.900594+0000","last_peered":"2024-04-25T16:27:02.900594+0000","last_clean":"2024-04-25T16:27:02.900594+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:02.900594+0000","last_undegraded":"2024-04-25T16:27:02.900594+0000","last_fullsized":"2024-04-25T16:27:02.900594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591187+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:01.591187+0000","last_peered":"2024-04-25T16:27:01.591187+0000","last_clean":"2024-04-25T16:27:01.591187+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:01.591187+0000","last_undegraded":"2024-04-25T16:27:01.591187+0000","last_fullsized":"2024-04-25T16:27:01.591187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161997+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:02.161997+0000","last_peered":"2024-04-25T16:27:02.161997+0000","last_clean":"2024-04-25T16:27:02.161997+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:02.161997+0000","last_undegraded":"2024-04-25T16:27:02.161997+0000","last_fullsized":"2024-04-25T16:27:02.161997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900648+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:02.900648+0000","last_peered":"2024-04-25T16:27:02.900648+0000","last_clean":"2024-04-25T16:27:02.900648+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:02.900648+0000","last_undegraded":"2024-04-25T16:27:02.900648+0000","last_fullsized":"2024-04-25T16:27:02.900648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901955+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:02.901955+0000","last_peered":"2024-04-25T16:27:02.901955+0000","last_clean":"2024-04-25T16:27:02.901955+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:02.901955+0000","last_undegraded":"2024-04-25T16:27:02.901955+0000","last_fullsized":"2024-04-25T16:27:02.901955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900856+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:02.900856+0000","last_peered":"2024-04-25T16:27:02.900856+0000","last_clean":"2024-04-25T16:27:02.900856+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:02.900856+0000","last_undegraded":"2024-04-25T16:27:02.900856+0000","last_fullsized":"2024-04-25T16:27:02.900856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901967+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:02.901967+0000","last_peered":"2024-04-25T16:27:02.901967+0000","last_clean":"2024-04-25T16:27:02.901967+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:02.901967+0000","last_undegraded":"2024-04-25T16:27:02.901967+0000","last_fullsized":"2024-04-25T16:27:02.901967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161498+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:02.161498+0000","last_peered":"2024-04-25T16:27:02.161498+0000","last_clean":"2024-04-25T16:27:02.161498+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:02.161498+0000","last_undegraded":"2024-04-25T16:27:02.161498+0000","last_fullsized":"2024-04-25T16:27:02.161498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900790+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:02.900790+0000","last_peered":"2024-04-25T16:27:02.900790+0000","last_clean":"2024-04-25T16:27:02.900790+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:02.900790+0000","last_undegraded":"2024-04-25T16:27:02.900790+0000","last_fullsized":"2024-04-25T16:27:02.900790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590401+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:01.590401+0000","last_peered":"2024-04-25T16:27:01.590401+0000","last_clean":"2024-04-25T16:27:01.590401+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:01.590401+0000","last_undegraded":"2024-04-25T16:27:01.590401+0000","last_fullsized":"2024-04-25T16:27:01.590401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589995+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:01.589995+0000","last_peered":"2024-04-25T16:27:01.589995+0000","last_clean":"2024-04-25T16:27:01.589995+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:01.589995+0000","last_undegraded":"2024-04-25T16:27:01.589995+0000","last_fullsized":"2024-04-25T16:27:01.589995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900320+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:02.900320+0000","last_peered":"2024-04-25T16:27:02.900320+0000","last_clean":"2024-04-25T16:27:02.900320+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:02.900320+0000","last_undegraded":"2024-04-25T16:27:02.900320+0000","last_fullsized":"2024-04-25T16:27:02.900320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900231+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:02.900231+0000","last_peered":"2024-04-25T16:27:02.900231+0000","last_clean":"2024-04-25T16:27:02.900231+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:02.900231+0000","last_undegraded":"2024-04-25T16:27:02.900231+0000","last_fullsized":"2024-04-25T16:27:02.900231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901358+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:02.901358+0000","last_peered":"2024-04-25T16:27:02.901358+0000","last_clean":"2024-04-25T16:27:02.901358+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:02.901358+0000","last_undegraded":"2024-04-25T16:27:02.901358+0000","last_fullsized":"2024-04-25T16:27:02.901358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589529+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:01.589529+0000","last_peered":"2024-04-25T16:27:01.589529+0000","last_clean":"2024-04-25T16:27:01.589529+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:01.589529+0000","last_undegraded":"2024-04-25T16:27:01.589529+0000","last_fullsized":"2024-04-25T16:27:01.589529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589509+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:01.589509+0000","last_peered":"2024-04-25T16:27:01.589509+0000","last_clean":"2024-04-25T16:27:01.589509+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:01.589509+0000","last_undegraded":"2024-04-25T16:27:01.589509+0000","last_fullsized":"2024-04-25T16:27:01.589509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584625+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:01.584625+0000","last_peered":"2024-04-25T16:27:01.584625+0000","last_clean":"2024-04-25T16:27:01.584625+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:01.584625+0000","last_undegraded":"2024-04-25T16:27:01.584625+0000","last_fullsized":"2024-04-25T16:27:01.584625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900487+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:02.900487+0000","last_peered":"2024-04-25T16:27:02.900487+0000","last_clean":"2024-04-25T16:27:02.900487+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:02.900487+0000","last_undegraded":"2024-04-25T16:27:02.900487+0000","last_fullsized":"2024-04-25T16:27:02.900487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161965+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:02.161965+0000","last_peered":"2024-04-25T16:27:02.161965+0000","last_clean":"2024-04-25T16:27:02.161965+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:02.161965+0000","last_undegraded":"2024-04-25T16:27:02.161965+0000","last_fullsized":"2024-04-25T16:27:02.161965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584525+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:01.584525+0000","last_peered":"2024-04-25T16:27:01.584525+0000","last_clean":"2024-04-25T16:27:01.584525+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:01.584525+0000","last_undegraded":"2024-04-25T16:27:01.584525+0000","last_fullsized":"2024-04-25T16:27:01.584525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902220+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:02.902220+0000","last_peered":"2024-04-25T16:27:02.902220+0000","last_clean":"2024-04-25T16:27:02.902220+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:02.902220+0000","last_undegraded":"2024-04-25T16:27:02.902220+0000","last_fullsized":"2024-04-25T16:27:02.902220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162131+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:02.162131+0000","last_peered":"2024-04-25T16:27:02.162131+0000","last_clean":"2024-04-25T16:27:02.162131+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:02.162131+0000","last_undegraded":"2024-04-25T16:27:02.162131+0000","last_fullsized":"2024-04-25T16:27:02.162131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900492+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:02.900492+0000","last_peered":"2024-04-25T16:27:02.900492+0000","last_clean":"2024-04-25T16:27:02.900492+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:02.900492+0000","last_undegraded":"2024-04-25T16:27:02.900492+0000","last_fullsized":"2024-04-25T16:27:02.900492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:07.253703+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:07.253703+0000","last_peered":"2024-04-25T16:27:07.253703+0000","last_clean":"2024-04-25T16:27:07.253703+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:07.253703+0000","last_undegraded":"2024-04-25T16:27:07.253703+0000","last_fullsized":"2024-04-25T16:27:07.253703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900859+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:02.900859+0000","last_peered":"2024-04-25T16:27:02.900859+0000","last_clean":"2024-04-25T16:27:02.900859+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:02.900859+0000","last_undegraded":"2024-04-25T16:27:02.900859+0000","last_fullsized":"2024-04-25T16:27:02.900859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590243+0000","last_change":"2024-04-25T16:26:56.570117+0000","last_active":"2024-04-25T16:27:01.590243+0000","last_peered":"2024-04-25T16:27:01.590243+0000","last_clean":"2024-04-25T16:27:01.590243+0000","last_became_active":"2024-04-25T16:26:56.569817+0000","last_became_peered":"2024-04-25T16:26:56.569817+0000","last_unstale":"2024-04-25T16:27:01.590243+0000","last_undegraded":"2024-04-25T16:27:01.590243+0000","last_fullsized":"2024-04-25T16:27:01.590243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:12.099635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900579+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:02.900579+0000","last_peered":"2024-04-25T16:27:02.900579+0000","last_clean":"2024-04-25T16:27:02.900579+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:02.900579+0000","last_undegraded":"2024-04-25T16:27:02.900579+0000","last_fullsized":"2024-04-25T16:27:02.900579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589933+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:01.589933+0000","last_peered":"2024-04-25T16:27:01.589933+0000","last_clean":"2024-04-25T16:27:01.589933+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:01.589933+0000","last_undegraded":"2024-04-25T16:27:01.589933+0000","last_fullsized":"2024-04-25T16:27:01.589933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162032+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:02.162032+0000","last_peered":"2024-04-25T16:27:02.162032+0000","last_clean":"2024-04-25T16:27:02.162032+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:02.162032+0000","last_undegraded":"2024-04-25T16:27:02.162032+0000","last_fullsized":"2024-04-25T16:27:02.162032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584281+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:01.584281+0000","last_peered":"2024-04-25T16:27:01.584281+0000","last_clean":"2024-04-25T16:27:01.584281+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:01.584281+0000","last_undegraded":"2024-04-25T16:27:01.584281+0000","last_fullsized":"2024-04-25T16:27:01.584281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161984+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:02.161984+0000","last_peered":"2024-04-25T16:27:02.161984+0000","last_clean":"2024-04-25T16:27:02.161984+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:02.161984+0000","last_undegraded":"2024-04-25T16:27:02.161984+0000","last_fullsized":"2024-04-25T16:27:02.161984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591106+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:01.591106+0000","last_peered":"2024-04-25T16:27:01.591106+0000","last_clean":"2024-04-25T16:27:01.591106+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:01.591106+0000","last_undegraded":"2024-04-25T16:27:01.591106+0000","last_fullsized":"2024-04-25T16:27:01.591106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584144+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:01.584144+0000","last_peered":"2024-04-25T16:27:01.584144+0000","last_clean":"2024-04-25T16:27:01.584144+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:01.584144+0000","last_undegraded":"2024-04-25T16:27:01.584144+0000","last_fullsized":"2024-04-25T16:27:01.584144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901211+0000","last_change":"2024-04-25T16:27:00.605248+0000","last_active":"2024-04-25T16:27:02.901211+0000","last_peered":"2024-04-25T16:27:02.901211+0000","last_clean":"2024-04-25T16:27:02.901211+0000","last_became_active":"2024-04-25T16:27:00.604746+0000","last_became_peered":"2024-04-25T16:27:00.604746+0000","last_unstale":"2024-04-25T16:27:02.901211+0000","last_undegraded":"2024-04-25T16:27:02.901211+0000","last_fullsized":"2024-04-25T16:27:02.901211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900313+0000","last_change":"2024-04-25T16:26:56.553599+0000","last_active":"2024-04-25T16:27:02.900313+0000","last_peered":"2024-04-25T16:27:02.900313+0000","last_clean":"2024-04-25T16:27:02.900313+0000","last_became_active":"2024-04-25T16:26:56.553304+0000","last_became_peered":"2024-04-25T16:26:56.553304+0000","last_unstale":"2024-04-25T16:27:02.900313+0000","last_undegraded":"2024-04-25T16:27:02.900313+0000","last_fullsized":"2024-04-25T16:27:02.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161615+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:02.161615+0000","last_peered":"2024-04-25T16:27:02.161615+0000","last_clean":"2024-04-25T16:27:02.161615+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:02.161615+0000","last_undegraded":"2024-04-25T16:27:02.161615+0000","last_fullsized":"2024-04-25T16:27:02.161615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591033+0000","last_change":"2024-04-25T16:27:00.587907+0000","last_active":"2024-04-25T16:27:01.591033+0000","last_peered":"2024-04-25T16:27:01.591033+0000","last_clean":"2024-04-25T16:27:01.591033+0000","last_became_active":"2024-04-25T16:27:00.587281+0000","last_became_peered":"2024-04-25T16:27:00.587281+0000","last_unstale":"2024-04-25T16:27:01.591033+0000","last_undegraded":"2024-04-25T16:27:01.591033+0000","last_fullsized":"2024-04-25T16:27:01.591033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584123+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:01.584123+0000","last_peered":"2024-04-25T16:27:01.584123+0000","last_clean":"2024-04-25T16:27:01.584123+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:01.584123+0000","last_undegraded":"2024-04-25T16:27:01.584123+0000","last_fullsized":"2024-04-25T16:27:01.584123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584701+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:01.584701+0000","last_peered":"2024-04-25T16:27:01.584701+0000","last_clean":"2024-04-25T16:27:01.584701+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:01.584701+0000","last_undegraded":"2024-04-25T16:27:01.584701+0000","last_fullsized":"2024-04-25T16:27:01.584701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591433+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:01.591433+0000","last_peered":"2024-04-25T16:27:01.591433+0000","last_clean":"2024-04-25T16:27:01.591433+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:01.591433+0000","last_undegraded":"2024-04-25T16:27:01.591433+0000","last_fullsized":"2024-04-25T16:27:01.591433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902190+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:02.902190+0000","last_peered":"2024-04-25T16:27:02.902190+0000","last_clean":"2024-04-25T16:27:02.902190+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:02.902190+0000","last_undegraded":"2024-04-25T16:27:02.902190+0000","last_fullsized":"2024-04-25T16:27:02.902190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900327+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:02.900327+0000","last_peered":"2024-04-25T16:27:02.900327+0000","last_clean":"2024-04-25T16:27:02.900327+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:02.900327+0000","last_undegraded":"2024-04-25T16:27:02.900327+0000","last_fullsized":"2024-04-25T16:27:02.900327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584664+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:01.584664+0000","last_peered":"2024-04-25T16:27:01.584664+0000","last_clean":"2024-04-25T16:27:01.584664+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:01.584664+0000","last_undegraded":"2024-04-25T16:27:01.584664+0000","last_fullsized":"2024-04-25T16:27:01.584664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589742+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:01.589742+0000","last_peered":"2024-04-25T16:27:01.589742+0000","last_clean":"2024-04-25T16:27:01.589742+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:01.589742+0000","last_undegraded":"2024-04-25T16:27:01.589742+0000","last_fullsized":"2024-04-25T16:27:01.589742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584547+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:01.584547+0000","last_peered":"2024-04-25T16:27:01.584547+0000","last_clean":"2024-04-25T16:27:01.584547+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:01.584547+0000","last_undegraded":"2024-04-25T16:27:01.584547+0000","last_fullsized":"2024-04-25T16:27:01.584547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589145+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:01.589145+0000","last_peered":"2024-04-25T16:27:01.589145+0000","last_clean":"2024-04-25T16:27:01.589145+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:01.589145+0000","last_undegraded":"2024-04-25T16:27:01.589145+0000","last_fullsized":"2024-04-25T16:27:01.589145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.899980+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:02.899980+0000","last_peered":"2024-04-25T16:27:02.899980+0000","last_clean":"2024-04-25T16:27:02.899980+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:02.899980+0000","last_undegraded":"2024-04-25T16:27:02.899980+0000","last_fullsized":"2024-04-25T16:27:02.899980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161855+0000","last_change":"2024-04-25T16:26:56.558625+0000","last_active":"2024-04-25T16:27:02.161855+0000","last_peered":"2024-04-25T16:27:02.161855+0000","last_clean":"2024-04-25T16:27:02.161855+0000","last_became_active":"2024-04-25T16:26:56.558322+0000","last_became_peered":"2024-04-25T16:26:56.558322+0000","last_unstale":"2024-04-25T16:27:02.161855+0000","last_undegraded":"2024-04-25T16:27:02.161855+0000","last_fullsized":"2024-04-25T16:27:02.161855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902073+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:02.902073+0000","last_peered":"2024-04-25T16:27:02.902073+0000","last_clean":"2024-04-25T16:27:02.902073+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:02.902073+0000","last_undegraded":"2024-04-25T16:27:02.902073+0000","last_fullsized":"2024-04-25T16:27:02.902073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584798+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:01.584798+0000","last_peered":"2024-04-25T16:27:01.584798+0000","last_clean":"2024-04-25T16:27:01.584798+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:01.584798+0000","last_undegraded":"2024-04-25T16:27:01.584798+0000","last_fullsized":"2024-04-25T16:27:01.584798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590378+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:01.590378+0000","last_peered":"2024-04-25T16:27:01.590378+0000","last_clean":"2024-04-25T16:27:01.590378+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:01.590378+0000","last_undegraded":"2024-04-25T16:27:01.590378+0000","last_fullsized":"2024-04-25T16:27:01.590378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584670+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:01.584670+0000","last_peered":"2024-04-25T16:27:01.584670+0000","last_clean":"2024-04-25T16:27:01.584670+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:01.584670+0000","last_undegraded":"2024-04-25T16:27:01.584670+0000","last_fullsized":"2024-04-25T16:27:01.584670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161594+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:02.161594+0000","last_peered":"2024-04-25T16:27:02.161594+0000","last_clean":"2024-04-25T16:27:02.161594+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:02.161594+0000","last_undegraded":"2024-04-25T16:27:02.161594+0000","last_fullsized":"2024-04-25T16:27:02.161594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584586+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:01.584586+0000","last_peered":"2024-04-25T16:27:01.584586+0000","last_clean":"2024-04-25T16:27:01.584586+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:01.584586+0000","last_undegraded":"2024-04-25T16:27:01.584586+0000","last_fullsized":"2024-04-25T16:27:01.584586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161810+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:02.161810+0000","last_peered":"2024-04-25T16:27:02.161810+0000","last_clean":"2024-04-25T16:27:02.161810+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:02.161810+0000","last_undegraded":"2024-04-25T16:27:02.161810+0000","last_fullsized":"2024-04-25T16:27:02.161810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901772+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:02.901772+0000","last_peered":"2024-04-25T16:27:02.901772+0000","last_clean":"2024-04-25T16:27:02.901772+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:02.901772+0000","last_undegraded":"2024-04-25T16:27:02.901772+0000","last_fullsized":"2024-04-25T16:27:02.901772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589540+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:01.589540+0000","last_peered":"2024-04-25T16:27:01.589540+0000","last_clean":"2024-04-25T16:27:01.589540+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:01.589540+0000","last_undegraded":"2024-04-25T16:27:01.589540+0000","last_fullsized":"2024-04-25T16:27:01.589540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900759+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:02.900759+0000","last_peered":"2024-04-25T16:27:02.900759+0000","last_clean":"2024-04-25T16:27:02.900759+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:02.900759+0000","last_undegraded":"2024-04-25T16:27:02.900759+0000","last_fullsized":"2024-04-25T16:27:02.900759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901860+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:02.901860+0000","last_peered":"2024-04-25T16:27:02.901860+0000","last_clean":"2024-04-25T16:27:02.901860+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:02.901860+0000","last_undegraded":"2024-04-25T16:27:02.901860+0000","last_fullsized":"2024-04-25T16:27:02.901860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162241+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:02.162241+0000","last_peered":"2024-04-25T16:27:02.162241+0000","last_clean":"2024-04-25T16:27:02.162241+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:02.162241+0000","last_undegraded":"2024-04-25T16:27:02.162241+0000","last_fullsized":"2024-04-25T16:27:02.162241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161412+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:02.161412+0000","last_peered":"2024-04-25T16:27:02.161412+0000","last_clean":"2024-04-25T16:27:02.161412+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:02.161412+0000","last_undegraded":"2024-04-25T16:27:02.161412+0000","last_fullsized":"2024-04-25T16:27:02.161412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900423+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:02.900423+0000","last_peered":"2024-04-25T16:27:02.900423+0000","last_clean":"2024-04-25T16:27:02.900423+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:02.900423+0000","last_undegraded":"2024-04-25T16:27:02.900423+0000","last_fullsized":"2024-04-25T16:27:02.900423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584351+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:01.584351+0000","last_peered":"2024-04-25T16:27:01.584351+0000","last_clean":"2024-04-25T16:27:01.584351+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:01.584351+0000","last_undegraded":"2024-04-25T16:27:01.584351+0000","last_fullsized":"2024-04-25T16:27:01.584351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590124+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:01.590124+0000","last_peered":"2024-04-25T16:27:01.590124+0000","last_clean":"2024-04-25T16:27:01.590124+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:01.590124+0000","last_undegraded":"2024-04-25T16:27:01.590124+0000","last_fullsized":"2024-04-25T16:27:01.590124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590011+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:01.590011+0000","last_peered":"2024-04-25T16:27:01.590011+0000","last_clean":"2024-04-25T16:27:01.590011+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:01.590011+0000","last_undegraded":"2024-04-25T16:27:01.590011+0000","last_fullsized":"2024-04-25T16:27:01.590011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590256+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:01.590256+0000","last_peered":"2024-04-25T16:27:01.590256+0000","last_clean":"2024-04-25T16:27:01.590256+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:01.590256+0000","last_undegraded":"2024-04-25T16:27:01.590256+0000","last_fullsized":"2024-04-25T16:27:01.590256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900847+0000","last_change":"2024-04-25T16:26:56.572739+0000","last_active":"2024-04-25T16:27:02.900847+0000","last_peered":"2024-04-25T16:27:02.900847+0000","last_clean":"2024-04-25T16:27:02.900847+0000","last_became_active":"2024-04-25T16:26:56.571251+0000","last_became_peered":"2024-04-25T16:26:56.571251+0000","last_unstale":"2024-04-25T16:27:02.900847+0000","last_undegraded":"2024-04-25T16:27:02.900847+0000","last_fullsized":"2024-04-25T16:27:02.900847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584005+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:01.584005+0000","last_peered":"2024-04-25T16:27:01.584005+0000","last_clean":"2024-04-25T16:27:01.584005+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:01.584005+0000","last_undegraded":"2024-04-25T16:27:01.584005+0000","last_fullsized":"2024-04-25T16:27:01.584005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900188+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:02.900188+0000","last_peered":"2024-04-25T16:27:02.900188+0000","last_clean":"2024-04-25T16:27:02.900188+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:02.900188+0000","last_undegraded":"2024-04-25T16:27:02.900188+0000","last_fullsized":"2024-04-25T16:27:02.900188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161603+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:02.161603+0000","last_peered":"2024-04-25T16:27:02.161603+0000","last_clean":"2024-04-25T16:27:02.161603+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:02.161603+0000","last_undegraded":"2024-04-25T16:27:02.161603+0000","last_fullsized":"2024-04-25T16:27:02.161603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161799+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:02.161799+0000","last_peered":"2024-04-25T16:27:02.161799+0000","last_clean":"2024-04-25T16:27:02.161799+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:02.161799+0000","last_undegraded":"2024-04-25T16:27:02.161799+0000","last_fullsized":"2024-04-25T16:27:02.161799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900832+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:02.900832+0000","last_peered":"2024-04-25T16:27:02.900832+0000","last_clean":"2024-04-25T16:27:02.900832+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:02.900832+0000","last_undegraded":"2024-04-25T16:27:02.900832+0000","last_fullsized":"2024-04-25T16:27:02.900832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901018+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:02.901018+0000","last_peered":"2024-04-25T16:27:02.901018+0000","last_clean":"2024-04-25T16:27:02.901018+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:02.901018+0000","last_undegraded":"2024-04-25T16:27:02.901018+0000","last_fullsized":"2024-04-25T16:27:02.901018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900715+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:02.900715+0000","last_peered":"2024-04-25T16:27:02.900715+0000","last_clean":"2024-04-25T16:27:02.900715+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:02.900715+0000","last_undegraded":"2024-04-25T16:27:02.900715+0000","last_fullsized":"2024-04-25T16:27:02.900715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":103933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":694813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:20.992 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T16:27:20.992 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T16:27:20.992 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T16:27:20.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T16:27:20.993 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T16:27:21.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.133+0000 7f55b9c42640 1 Processor -- start 2024-04-25T16:27:21.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.133+0000 7f55b9c42640 1 -- start start 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b9c42640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55b40b5230 0x7f55b4160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b9c42640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b9c42640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b40b5a30 con 0x7f55b4164b60 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b9c42640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f55b406ca10 con 0x7f55b40b5230 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b9c42640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f55b40d5610 con 0x7f55b4164210 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b37fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55b40b5230 0x7f55b4160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b2ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b37fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55b40b5230 0x7f55b4160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46382/0 (socket says 172.21.15.94:46382) 2024-04-25T16:27:21.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b37fe640 1 -- 172.21.15.94:0/4232751978 learned_addr learned my addr 172.21.15.94:0/4232751978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:21.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.134+0000 7f55b3fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f55b4164b60 0x7f55b40d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:56928/0 (socket says 172.21.15.94:56928) 2024-04-25T16:27:21.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.135+0000 7f55b37fe640 1 -- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 msgr2=0x7f55b4164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:21.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.135+0000 7f55b37fe640 1 --2- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4164620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.135+0000 7f55b37fe640 1 -- 172.21.15.94:0/4232751978 >> v1:172.21.15.138:6789/0 conn(0x7f55b4164b60 legacy=0x7f55b40d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.135+0000 7f55b37fe640 1 -- 172.21.15.94:0/4232751978 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40d58f0 con 0x7f55b40b5230 2024-04-25T16:27:21.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.135+0000 7f55b2ffd640 1 --2- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4164620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:21.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.135+0000 7f55b37fe640 1 --2- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b40b5230 0x7f55b4160790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009870 tx=0x7f55a402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb77b89854424b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b27fc640 1 -- 172.21.15.94:0/4232751978 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4037070 con 0x7f55b40b5230 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b27fc640 1 -- 172.21.15.94:0/4232751978 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a4032440 con 0x7f55b40b5230 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b27fc640 1 -- 172.21.15.94:0/4232751978 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4032810 con 0x7f55b40b5230 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 -- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b40b5230 msgr2=0x7f55b4160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 --2- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b40b5230 0x7f55b4160790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009870 tx=0x7f55a402a1c0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 -- 172.21.15.94:0/4232751978 shutdown_connections 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 --2- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4164620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 --2- 172.21.15.94:0/4232751978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b40b5230 0x7f55b4160790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 -- 172.21.15.94:0/4232751978 >> 172.21.15.94:0/4232751978 conn(0x7f55b40d3570 msgr2=0x7f55b40b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 -- 172.21.15.94:0/4232751978 shutdown_connections 2024-04-25T16:27:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.136+0000 7f55b9c42640 1 -- 172.21.15.94:0/4232751978 wait complete. 2024-04-25T16:27:21.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.138+0000 7f55b9c42640 1 Processor -- start 2024-04-25T16:27:21.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.138+0000 7f55b9c42640 1 -- start start 2024-04-25T16:27:21.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 0x7f55b41f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4162e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b4164b60 0x7f55b415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4164f70 con 0x7f55b40b5230 2024-04-25T16:27:21.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55b406deb0 con 0x7f55b4164b60 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f55b40b96f0 con 0x7f55b4164210 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 0x7f55b41f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 0x7f55b41f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35850/0 (socket says 172.21.15.94:35850) 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 -- 172.21.15.94:0/40091461 learned_addr learned my addr 172.21.15.94:0/40091461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b3fff640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b4164b60 0x7f55b415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b2ffd640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 -- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 msgr2=0x7f55b4162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 -- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b4164b60 msgr2=0x7f55b415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b4164b60 0x7f55b415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 -- 172.21.15.94:0/40091461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40c9e20 con 0x7f55b40b5230 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b37fe640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 0x7f55b41f62d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000c00 tx=0x7f55a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4037040 con 0x7f55b40b5230 2024-04-25T16:27:21.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a4042070 con 0x7f55b40b5230 2024-04-25T16:27:21.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b40ca100 con 0x7f55b40b5230 2024-04-25T16:27:21.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.140+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4030880 con 0x7f55b40b5230 2024-04-25T16:27:21.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.141+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b40cbc60 con 0x7f55b40b5230 2024-04-25T16:27:21.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.141+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b406a180 con 0x7f55b40b5230 2024-04-25T16:27:21.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.142+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f55a402ad10 con 0x7f55b40b5230 2024-04-25T16:27:21.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.143+0000 7f559bfff640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f55a00744b0 0x7f55a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.143+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f55a40b5ad0 con 0x7f55b40b5230 2024-04-25T16:27:21.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.143+0000 7f55b2ffd640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f55a00744b0 0x7f55a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.143+0000 7f55b2ffd640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f55a00744b0 0x7f55a00769c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55b40b76d0 tx=0x7f55a8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.147+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55a407f850 con 0x7f55b40b5230 2024-04-25T16:27:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.373+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f55b40b7340 con 0x7f55b40b5230 2024-04-25T16:27:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.377+0000 7f559bfff640 1 -- 172.21.15.94:0/40091461 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186742 (secure 0 0 0) 0x7f55a4083b50 con 0x7f55b40b5230 2024-04-25T16:27:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f55a00744b0 msgr2=0x7f55a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f55a00744b0 0x7f55a00769c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55b40b76d0 tx=0x7f55a8009190 comp rx=0 tx=0).stop 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 msgr2=0x7f55b41f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 0x7f55b41f62d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000c00 tx=0x7f55a40046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 shutdown_connections 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f55a00744b0 0x7f55a00769c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55b4164b60 0x7f55b415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55b4164210 0x7f55b4162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 --2- 172.21.15.94:0/40091461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55b40b5230 0x7f55b41f62d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 >> 172.21.15.94:0/40091461 conn(0x7f55b40d3570 msgr2=0x7f55b40b6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 shutdown_connections 2024-04-25T16:27:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.379+0000 7f55b9c42640 1 -- 172.21.15.94:0/40091461 wait complete. 2024-04-25T16:27:21.391 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2607833826},{"type":"v1","addr":"172.21.15.94:6833","nonce":2607833826}]},"active_addr":"172.21.15.94:6833/2607833826","active_change":"2024-04-25T16:26:35.287169+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1802582322}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":4215409881}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2789075255}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":809993238}]}]} 2024-04-25T16:27:21.394 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T16:27:21.394 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T16:27:21.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:21.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.500+0000 7fb3cde92640 1 Processor -- start 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.500+0000 7fb3cde92640 1 -- start start 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3cde92640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3c80d2090 0x7fb3c80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3cde92640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3c80d11b0 0x7fb3c80cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3cde92640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c80d24a0 con 0x7fb3c808e080 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3cde92640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c80d2720 con 0x7fb3c80d2090 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3cde92640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c80d2a00 con 0x7fb3c80d11b0 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3c80d2090 0x7fb3c80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3c80d2090 0x7fb3c80b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55840/0 (socket says 172.21.15.94:55840) 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 -- 172.21.15.94:0/3336930722 learned_addr learned my addr 172.21.15.94:0/3336930722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c7fff640 1 --2- 172.21.15.94:0/3336930722 >> v2:172.21.15.138:3300/0 conn(0x7fb3c80d11b0 0x7fb3c80cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 -- 172.21.15.94:0/3336930722 >> v2:172.21.15.138:3300/0 conn(0x7fb3c80d11b0 msgr2=0x7fb3c80cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 --2- 172.21.15.94:0/3336930722 >> v2:172.21.15.138:3300/0 conn(0x7fb3c80d11b0 0x7fb3c80cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 -- 172.21.15.94:0/3336930722 >> v1:172.21.15.138:6789/0 conn(0x7fb3c808e080 legacy=0x7fb3c80d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:21.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.502+0000 7fb3c77fe640 1 -- 172.21.15.94:0/3336930722 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c80d2ce0 con 0x7fb3c80d2090 2024-04-25T16:27:21.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.503+0000 7fb3c77fe640 1 --2- 172.21.15.94:0/3336930722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80d2090 0x7fb3c80b3350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b3d50 tx=0x7fb3b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cc53ed08054f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.503+0000 7fb3c67fc640 1 -- 172.21.15.94:0/3336930722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8004030 con 0x7fb3c80d2090 2024-04-25T16:27:21.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.503+0000 7fb3c67fc640 1 -- 172.21.15.94:0/3336930722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b80041d0 con 0x7fb3c80d2090 2024-04-25T16:27:21.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.503+0000 7fb3c67fc640 1 -- 172.21.15.94:0/3336930722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8003610 con 0x7fb3c80d2090 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 -- 172.21.15.94:0/3336930722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80d2090 msgr2=0x7fb3c80b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 --2- 172.21.15.94:0/3336930722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80d2090 0x7fb3c80b3350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b3d50 tx=0x7fb3b802a180 comp rx=0 tx=0).stop 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 -- 172.21.15.94:0/3336930722 shutdown_connections 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 --2- 172.21.15.94:0/3336930722 >> v2:172.21.15.138:3300/0 conn(0x7fb3c80d11b0 0x7fb3c80cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 --2- 172.21.15.94:0/3336930722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80d2090 0x7fb3c80b3350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 -- 172.21.15.94:0/3336930722 >> 172.21.15.94:0/3336930722 conn(0x7fb3c80d4090 msgr2=0x7fb3c80d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 -- 172.21.15.94:0/3336930722 shutdown_connections 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.504+0000 7fb3cde92640 1 -- 172.21.15.94:0/3336930722 wait complete. 2024-04-25T16:27:21.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.505+0000 7fb3cde92640 1 Processor -- start 2024-04-25T16:27:21.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.506+0000 7fb3cde92640 1 -- start start 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3cde92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c808e080 0x7fb3c80d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3cde92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3c80d2090 0x7fb3c81590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3cde92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 0x7fb3c80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3cde92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80ced50 con 0x7fb3c81f88b0 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3cde92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c808e490 con 0x7fb3c808e080 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3cde92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c80b4470 con 0x7fb3c80d2090 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 0x7fb3c80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 0x7fb3c80b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55842/0 (socket says 172.21.15.94:55842) 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 -- 172.21.15.94:0/59391481 learned_addr learned my addr 172.21.15.94:0/59391481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c6ffd640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3c80d2090 0x7fb3c81590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c77fe640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c808e080 0x7fb3c80d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 -- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3c80d2090 msgr2=0x7fb3c81590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3c80d2090 0x7fb3c81590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 -- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c808e080 msgr2=0x7fb3c80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c808e080 0x7fb3c80d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.507+0000 7fb3c7fff640 1 -- 172.21.15.94:0/59391481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c8159a10 con 0x7fb3c81f88b0 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.508+0000 7fb3c7fff640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 0x7fb3c80b23f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0007d80 tx=0x7fb3c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.508+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0014420 con 0x7fb3c81f88b0 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.508+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0019070 con 0x7fb3c81f88b0 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.508+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c80cc6f0 con 0x7fb3c81f88b0 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.508+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0014800 con 0x7fb3c81f88b0 2024-04-25T16:27:21.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.508+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c80cccb0 con 0x7fb3c81f88b0 2024-04-25T16:27:21.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.509+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c806a180 con 0x7fb3c81f88b0 2024-04-25T16:27:21.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.509+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb3c00149a0 con 0x7fb3c81f88b0 2024-04-25T16:27:21.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.510+0000 7fb3affff640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3b40744b0 0x7fb3b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.510+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb3c0095c60 con 0x7fb3c81f88b0 2024-04-25T16:27:21.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.510+0000 7fb3c77fe640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3b40744b0 0x7fb3b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.510+0000 7fb3c77fe640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3b40744b0 0x7fb3b40769c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8033000 tx=0x7fb3b802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.514+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3c005fa50 con 0x7fb3c81f88b0 2024-04-25T16:27:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.693+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3c81f8cd0 con 0x7fb3c81f88b0 2024-04-25T16:27:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.695+0000 7fb3affff640 1 -- 172.21.15.94:0/59391481 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fb3c0063d50 con 0x7fb3c81f88b0 2024-04-25T16:27:21.695 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:21.696 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3b40744b0 msgr2=0x7fb3b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3b40744b0 0x7fb3b40769c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8033000 tx=0x7fb3b802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 msgr2=0x7fb3c80b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 0x7fb3c80b23f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0007d80 tx=0x7fb3c000e4b0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 shutdown_connections 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb3b40744b0 0x7fb3b40769c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c81f88b0 0x7fb3c80b23f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3c80d2090 0x7fb3c81590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 --2- 172.21.15.94:0/59391481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c808e080 0x7fb3c80d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 >> 172.21.15.94:0/59391481 conn(0x7fb3c80d4090 msgr2=0x7fb3c80b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 shutdown_connections 2024-04-25T16:27:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.697+0000 7fb3cde92640 1 -- 172.21.15.94:0/59391481 wait complete. 2024-04-25T16:27:21.709 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T16:27:21.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.816+0000 7fdd24d4d640 1 Processor -- start 2024-04-25T16:27:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.816+0000 7fdd24d4d640 1 -- start start 2024-04-25T16:27:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.817+0000 7fdd24d4d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd200b3570 0x7fdd200b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.817+0000 7fdd24d4d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdd200d11b0 0x7fdd200d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.817+0000 7fdd24d4d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd200b44d0 con 0x7fdd2006ef10 2024-04-25T16:27:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.817+0000 7fdd24d4d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fdd200b50e0 con 0x7fdd200d11b0 2024-04-25T16:27:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.817+0000 7fdd24d4d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd200b42a0 con 0x7fdd200b3570 2024-04-25T16:27:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdd200d11b0 0x7fdd200d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd15d9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdd2006ef10 0x7fdd200d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56180/0 (socket says 172.21.15.94:56180) 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd15d9b640 1 -- 172.21.15.94:0/1780254440 learned_addr learned my addr 172.21.15.94:0/1780254440 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1e59c640 1 --2- 172.21.15.94:0/1780254440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd200b3570 0x7fdd200b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1dd9b640 1 -- 172.21.15.94:0/1780254440 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594123026 0 0) 0x7fdd200b44d0 con 0x7fdd2006ef10 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1dd9b640 1 -- 172.21.15.94:0/1780254440 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd0c003520 con 0x7fdd2006ef10 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1ed9d640 1 -- 172.21.15.94:0/1780254440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd200b3570 msgr2=0x7fdd200b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1ed9d640 1 --2- 172.21.15.94:0/1780254440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd200b3570 0x7fdd200b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1ed9d640 1 -- 172.21.15.94:0/1780254440 >> v1:172.21.15.94:6789/0 conn(0x7fdd2006ef10 legacy=0x7fdd200d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.818+0000 7fdd1ed9d640 1 -- 172.21.15.94:0/1780254440 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd200b47b0 con 0x7fdd200d11b0 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.819+0000 7fdd1ed9d640 1 --2- 172.21.15.94:0/1780254440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200d11b0 0x7fdd200d2590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009d90 tx=0x7fdd1802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0356fb6c7a0b86e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.819+0000 7fdd1dd9b640 1 -- 172.21.15.94:0/1780254440 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007740 con 0x7fdd200d11b0 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.819+0000 7fdd1dd9b640 1 -- 172.21.15.94:0/1780254440 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd180078e0 con 0x7fdd200d11b0 2024-04-25T16:27:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.819+0000 7fdd1dd9b640 1 -- 172.21.15.94:0/1780254440 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007c90 con 0x7fdd200d11b0 2024-04-25T16:27:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.819+0000 7fdd24d4d640 1 -- 172.21.15.94:0/1780254440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200d11b0 msgr2=0x7fdd200d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.819+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/1780254440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200d11b0 0x7fdd200d2590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009d90 tx=0x7fdd1802a6f0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.820+0000 7fdd24d4d640 1 -- 172.21.15.94:0/1780254440 shutdown_connections 2024-04-25T16:27:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.820+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/1780254440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200d11b0 0x7fdd200d2590 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.820+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/1780254440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd200b3570 0x7fdd200b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.820+0000 7fdd24d4d640 1 -- 172.21.15.94:0/1780254440 >> 172.21.15.94:0/1780254440 conn(0x7fdd200d4090 msgr2=0x7fdd200d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.820+0000 7fdd24d4d640 1 -- 172.21.15.94:0/1780254440 shutdown_connections 2024-04-25T16:27:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.820+0000 7fdd24d4d640 1 -- 172.21.15.94:0/1780254440 wait complete. 2024-04-25T16:27:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.822+0000 7fdd24d4d640 1 Processor -- start 2024-04-25T16:27:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.822+0000 7fdd24d4d640 1 -- start start 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd24d4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2006ef10 0x7fdd20159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd24d4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 0x7fdd2015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd24d4d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd201f6c00 0x7fdd200b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd24d4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2006f320 con 0x7fdd200b3570 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd24d4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd200b4d70 con 0x7fdd2006ef10 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd24d4d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd200d2ad0 con 0x7fdd201f6c00 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 0x7fdd2015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 0x7fdd2015b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55858/0 (socket says 172.21.15.94:55858) 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 -- 172.21.15.94:0/4229969689 learned_addr learned my addr 172.21.15.94:0/4229969689 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd1e59c640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2006ef10 0x7fdd20159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd1ed9d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd201f6c00 0x7fdd200b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 -- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd201f6c00 msgr2=0x7fdd200b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd201f6c00 0x7fdd200b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 -- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2006ef10 msgr2=0x7fdd20159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2006ef10 0x7fdd20159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd15d9b640 1 -- 172.21.15.94:0/4229969689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2015d1c0 con 0x7fdd200b3570 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd1e59c640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2006ef10 0x7fdd20159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.823+0000 7fdd1ed9d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd201f6c00 0x7fdd200b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.824+0000 7fdd15d9b640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 0x7fdd2015b0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c3c0 tx=0x7fdd0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.824+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd200b3570 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.824+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08005ce0 con 0x7fdd200b3570 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.824+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd20162650 con 0x7fdd200b3570 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.824+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080072f0 con 0x7fdd200b3570 2024-04-25T16:27:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.824+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd20162c10 con 0x7fdd200b3570 2024-04-25T16:27:21.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.825+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2006a180 con 0x7fdd200b3570 2024-04-25T16:27:21.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.825+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd0801a070 con 0x7fdd200b3570 2024-04-25T16:27:21.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.826+0000 7fdd177fe640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdd000744b0 0x7fdd000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.826+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd080948b0 con 0x7fdd200b3570 2024-04-25T16:27:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.826+0000 7fdd1e59c640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdd000744b0 0x7fdd000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.826+0000 7fdd1e59c640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdd000744b0 0x7fdd000769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ae0 tx=0x7fdd10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:21.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:21.831+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd0805ed90 con 0x7fdd200b3570 2024-04-25T16:27:22.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.006+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd200b9970 con 0x7fdd200b3570 2024-04-25T16:27:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.007+0000 7fdd177fe640 1 -- 172.21.15.94:0/4229969689 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fdd0805e690 con 0x7fdd200b3570 2024-04-25T16:27:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:22.008 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdd000744b0 msgr2=0x7fdd000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdd000744b0 0x7fdd000769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ae0 tx=0x7fdd10008040 comp rx=0 tx=0).stop 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 msgr2=0x7fdd2015b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 0x7fdd2015b0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c3c0 tx=0x7fdd0800c880 comp rx=0 tx=0).stop 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 shutdown_connections 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdd000744b0 0x7fdd000769c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd201f6c00 0x7fdd200b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd200b3570 0x7fdd2015b0c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 --2- 172.21.15.94:0/4229969689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2006ef10 0x7fdd20159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 >> 172.21.15.94:0/4229969689 conn(0x7fdd200d4090 msgr2=0x7fdd200b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 shutdown_connections 2024-04-25T16:27:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.009+0000 7fdd24d4d640 1 -- 172.21.15.94:0/4229969689 wait complete. 2024-04-25T16:27:22.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T16:27:22.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T16:27:22.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T16:27:22.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T16:27:22.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T16:27:22.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T16:27:22.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T16:27:22.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T16:27:22.185 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:27:22.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:27:22.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.234+0000 7fce5fcd8640 1 Processor -- start 2024-04-25T16:27:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.234+0000 7fce5fcd8640 1 -- start start 2024-04-25T16:27:22.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.235+0000 7f32e605b640 1 Processor -- start 2024-04-25T16:27:22.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.236+0000 7f32e605b640 1 -- start start 2024-04-25T16:27:22.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.235+0000 7f1719be0640 1 Processor -- start 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.236+0000 7fce5fcd8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fce580b4d10 0x7fce580b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.236+0000 7fce5fcd8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce5815ca70 0x7fce5815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.236+0000 7fce5fcd8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580afe30 con 0x7fce580b08b0 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.236+0000 7fce5fcd8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fce580b0420 con 0x7fce580b4d10 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.236+0000 7fce5fcd8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fce5815d4f0 con 0x7fce5815ca70 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fce580b4d10 0x7fce580b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fce580b4d10 0x7fce580b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55872/0 (socket says 172.21.15.94:55872) 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 -- 172.21.15.94:0/2611435369 learned_addr learned my addr 172.21.15.94:0/2611435369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5d273640 1 --2- 172.21.15.94:0/2611435369 >> v2:172.21.15.138:3300/0 conn(0x7fce5815ca70 0x7fce5815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 -- 172.21.15.94:0/2611435369 >> v2:172.21.15.138:3300/0 conn(0x7fce5815ca70 msgr2=0x7fce5815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 --2- 172.21.15.94:0/2611435369 >> v2:172.21.15.138:3300/0 conn(0x7fce5815ca70 0x7fce5815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 -- 172.21.15.94:0/2611435369 >> v1:172.21.15.94:6789/0 conn(0x7fce580b08b0 legacy=0x7fce580b0cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:27:22.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.237+0000 7fce5da74640 1 -- 172.21.15.94:0/2611435369 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5815d7d0 con 0x7fce580b4d10 2024-04-25T16:27:22.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.235+0000 7f1719be0640 1 -- start start 2024-04-25T16:27:22.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1719be0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1714168030 0x7f17140c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1719be0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17140b2d10 0x7f17140cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e605b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f32d80092a0 0x7f32d8050d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e605b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d8055860 con 0x7f32d8054560 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e605b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d80080b0 con 0x7f32d80e04d0 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e605b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f32d8008700 con 0x7f32d80092a0 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e5059640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f32d8054560 0x7f32d80394f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56194/0 (socket says 172.21.15.94:56194) 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e5059640 1 -- 172.21.15.94:0/1127050536 learned_addr learned my addr 172.21.15.94:0/1127050536 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1719be0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17140c65b0 con 0x7f17140c61a0 2024-04-25T16:27:22.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f32e585a640 1 --2- 172.21.15.94:0/1127050536 >> v2:172.21.15.138:3300/0 conn(0x7f32d80092a0 0x7f32d8050d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1719be0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f17140d3a90 con 0x7f1714168030 2024-04-25T16:27:22.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1719be0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f17140d2cd0 con 0x7f17140b2d10 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f32e585a640 1 -- 172.21.15.94:0/1127050536 >> v1:172.21.15.138:6789/0 conn(0x7f32d80e04d0 legacy=0x7f32d8008b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f32e585a640 1 -- 172.21.15.94:0/1127050536 >> v1:172.21.15.94:6789/0 conn(0x7f32d8054560 legacy=0x7f32d80394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f32e585a640 1 -- 172.21.15.94:0/1127050536 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d80e7360 con 0x7f32d80092a0 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7fce5da74640 1 --2- 172.21.15.94:0/2611435369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b4d10 0x7fce580b5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce580752d0 tx=0x7fce5402a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c51fd2e04796bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f17193df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17140b2d10 0x7f17140cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1718bde640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1714168030 0x7f17140c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1718bde640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1714168030 0x7f17140c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55880/0 (socket says 172.21.15.94:55880) 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.238+0000 7f1718bde640 1 -- 172.21.15.94:0/47171569 learned_addr learned my addr 172.21.15.94:0/47171569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f1718bde640 1 -- 172.21.15.94:0/47171569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17140b2d10 msgr2=0x7f17140cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f1718bde640 1 --2- 172.21.15.94:0/47171569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17140b2d10 0x7f17140cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f1718bde640 1 -- 172.21.15.94:0/47171569 >> v1:172.21.15.138:6789/0 conn(0x7f17140c61a0 legacy=0x7f17140b2600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.239+0000 7f1718bde640 1 -- 172.21.15.94:0/47171569 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17140d2fb0 con 0x7f1714168030 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.240+0000 7fce5ca72640 1 -- 172.21.15.94:0/2611435369 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54037070 con 0x7fce580b4d10 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.240+0000 7fce5ca72640 1 -- 172.21.15.94:0/2611435369 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce54032440 con 0x7fce580b4d10 2024-04-25T16:27:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.240+0000 7fce5ca72640 1 -- 172.21.15.94:0/2611435369 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54032810 con 0x7fce580b4d10 2024-04-25T16:27:22.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.240+0000 7f1718bde640 1 --2- 172.21.15.94:0/47171569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714168030 0x7f17140c5c60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1708009870 tx=0x7f170802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7184a6edb2020ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.240+0000 7f32e585a640 1 --2- 172.21.15.94:0/1127050536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 0x7f32d8050d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f32e0052040 tx=0x7f32e00a7700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155e588d749658f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.240+0000 7f1cf6d3c640 1 Processor -- start 2024-04-25T16:27:22.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7f17137fe640 1 -- 172.21.15.94:0/47171569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708037070 con 0x7f1714168030 2024-04-25T16:27:22.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7f17137fe640 1 -- 172.21.15.94:0/47171569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1708032440 con 0x7f1714168030 2024-04-25T16:27:22.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fe63ffff640 1 Processor -- start 2024-04-25T16:27:22.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7f17137fe640 1 -- 172.21.15.94:0/47171569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708032810 con 0x7f1714168030 2024-04-25T16:27:22.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fe63ffff640 1 -- start start 2024-04-25T16:27:22.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2611435369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b4d10 msgr2=0x7fce580b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 --2- 172.21.15.94:0/2611435369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b4d10 0x7fce580b5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce580752d0 tx=0x7fce5402a400 comp rx=0 tx=0).stop 2024-04-25T16:27:22.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2611435369 shutdown_connections 2024-04-25T16:27:22.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 --2- 172.21.15.94:0/2611435369 >> v2:172.21.15.138:3300/0 conn(0x7fce5815ca70 0x7fce5815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 --2- 172.21.15.94:0/2611435369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b4d10 0x7fce580b5120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2611435369 >> 172.21.15.94:0/2611435369 conn(0x7fce580733f0 msgr2=0x7fce58073800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7f1cf6d3c640 1 -- start start 2024-04-25T16:27:22.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f32d7fff640 1 -- 172.21.15.94:0/1127050536 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b3070 con 0x7f32d80092a0 2024-04-25T16:27:22.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f32d7fff640 1 -- 172.21.15.94:0/1127050536 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e00ae600 con 0x7f32d80092a0 2024-04-25T16:27:22.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f32d7fff640 1 -- 172.21.15.94:0/1127050536 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00aea50 con 0x7f32d80092a0 2024-04-25T16:27:22.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2611435369 shutdown_connections 2024-04-25T16:27:22.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.241+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2611435369 wait complete. 2024-04-25T16:27:22.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f1719be0640 1 -- 172.21.15.94:0/47171569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714168030 msgr2=0x7f17140c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f1719be0640 1 --2- 172.21.15.94:0/47171569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714168030 0x7f17140c5c60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1708009870 tx=0x7f170802a180 comp rx=0 tx=0).stop 2024-04-25T16:27:22.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f1719be0640 1 -- 172.21.15.94:0/47171569 shutdown_connections 2024-04-25T16:27:22.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f1719be0640 1 --2- 172.21.15.94:0/47171569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17140b2d10 0x7f17140cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.242+0000 7f1719be0640 1 --2- 172.21.15.94:0/47171569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714168030 0x7f17140c5c60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 0x7fe6400aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 0x7fe6400b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63ffff640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64015db60 con 0x7fe6400cdf70 2024-04-25T16:27:22.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63ffff640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe64015e490 con 0x7fe64015f930 2024-04-25T16:27:22.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63ffff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe64015de40 con 0x7fe64015b0d0 2024-04-25T16:27:22.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1719be0640 1 -- 172.21.15.94:0/47171569 >> 172.21.15.94:0/47171569 conn(0x7f171409b9b0 msgr2=0x7f171409bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1719be0640 1 -- 172.21.15.94:0/47171569 shutdown_connections 2024-04-25T16:27:22.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f32e605b640 1 -- 172.21.15.94:0/1127050536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 msgr2=0x7f32d8050d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f32e605b640 1 --2- 172.21.15.94:0/1127050536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 0x7f32d8050d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f32e0052040 tx=0x7f32e00a7700 comp rx=0 tx=0).stop 2024-04-25T16:27:22.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f48852f9640 1 Processor -- start 2024-04-25T16:27:22.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 0x7fe6400b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 0x7fe6400aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f32e605b640 1 -- 172.21.15.94:0/1127050536 shutdown_connections 2024-04-25T16:27:22.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f32e605b640 1 --2- 172.21.15.94:0/1127050536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 0x7f32d8050d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f32e605b640 1 -- 172.21.15.94:0/1127050536 >> 172.21.15.94:0/1127050536 conn(0x7f32d8055e00 msgr2=0x7f32d8056210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1cf6d3c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1ce8048300 0x7f1ce80e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1cf6d3c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce8006130 con 0x7f1ce8047db0 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1cf6d3c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce804ffa0 con 0x7f1ce8034060 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1cf6d3c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce804f690 con 0x7f1ce8048300 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1cf5d3a640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1ce8047db0 0x7f1ce8033950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56210/0 (socket says 172.21.15.94:56210) 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1cf5d3a640 1 -- 172.21.15.94:0/2414793717 learned_addr learned my addr 172.21.15.94:0/2414793717 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 0x7fe6400aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:59360/0 (socket says 172.21.15.94:59360) 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63effd640 1 -- 172.21.15.94:0/2227569531 learned_addr learned my addr 172.21.15.94:0/2227569531 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7fe63f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 0x7fe6400b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55896/0 (socket says 172.21.15.94:55896) 2024-04-25T16:27:22.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2414793717 >> v2:172.21.15.94:3300/0 conn(0x7f1ce8048300 0x7f1ce80e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f48852f9640 1 -- start start 2024-04-25T16:27:22.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f32e605b640 1 -- 172.21.15.94:0/1127050536 shutdown_connections 2024-04-25T16:27:22.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f1cf4d38640 1 -- 172.21.15.94:0/2414793717 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743444455 0 0) 0x7f1ce8006130 con 0x7f1ce8047db0 2024-04-25T16:27:22.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.243+0000 7f1719be0640 1 -- 172.21.15.94:0/47171569 wait complete. 2024-04-25T16:27:22.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f1cf4d38640 1 -- 172.21.15.94:0/2414793717 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cdc003520 con 0x7f1ce8047db0 2024-04-25T16:27:22.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fe63effd640 1 -- 172.21.15.94:0/2227569531 >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 msgr2=0x7fe6400b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fe63effd640 1 --2- 172.21.15.94:0/2227569531 >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 0x7fe6400b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f1cf653b640 1 -- 172.21.15.94:0/2414793717 >> v1:172.21.15.138:6789/0 conn(0x7f1ce8034060 legacy=0x7f1ce80065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fce5fcd8640 1 Processor -- start 2024-04-25T16:27:22.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fe63effd640 1 -- 172.21.15.94:0/2227569531 >> v1:172.21.15.138:6789/0 conn(0x7fe6400cdf70 legacy=0x7fe6400aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fe63effd640 1 -- 172.21.15.94:0/2227569531 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64015e120 con 0x7fe64015b0d0 2024-04-25T16:27:22.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fe63f7fe640 1 --2- 172.21.15.94:0/2227569531 >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 0x7fe6400b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:22.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f1cf653b640 1 -- 172.21.15.94:0/2414793717 >> v1:172.21.15.94:6789/0 conn(0x7f1ce8047db0 legacy=0x7f1ce8033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7f1cf653b640 1 -- 172.21.15.94:0/2414793717 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce8050280 con 0x7f1ce8048300 2024-04-25T16:27:22.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.244+0000 7fdb02bb4640 1 Processor -- start 2024-04-25T16:27:22.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7fdb02bb4640 1 -- start start 2024-04-25T16:27:22.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7f48852f9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48800b6ac0 0x7f48801658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7f48852f9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f488006cf50 con 0x7f48800b66b0 2024-04-25T16:27:22.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7f48852f9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48800b7730 con 0x7f48800b7200 2024-04-25T16:27:22.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7f48852f9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f488006d820 con 0x7f48800b6ac0 2024-04-25T16:27:22.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f487effd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f48800b7200 0x7f48800b5fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:51632/0 (socket says 172.21.15.94:51632) 2024-04-25T16:27:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7fe63effd640 1 --2- 172.21.15.94:0/2227569531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 0x7fe6400aff10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe63400b920 tx=0x7fe63402a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea0baff3a294b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7f32e605b640 1 -- 172.21.15.94:0/1127050536 wait complete. 2024-04-25T16:27:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f487effd640 1 -- 172.21.15.94:0/1353604341 learned_addr learned my addr 172.21.15.94:0/1353604341 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:22.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f1719be0640 1 Processor -- start 2024-04-25T16:27:22.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f1719be0640 1 -- start start 2024-04-25T16:27:22.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f4c19422640 1 Processor -- start 2024-04-25T16:27:22.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7fe63dffb640 1 -- 172.21.15.94:0/2227569531 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63402f510 con 0x7fe64015b0d0 2024-04-25T16:27:22.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7fe63dffb640 1 -- 172.21.15.94:0/2227569531 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63402f6b0 con 0x7fe64015b0d0 2024-04-25T16:27:22.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7fe63dffb640 1 -- 172.21.15.94:0/2227569531 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63402fa80 con 0x7fe64015b0d0 2024-04-25T16:27:22.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f487f7fe640 1 --2- 172.21.15.94:0/1353604341 >> v2:172.21.15.138:3300/0 conn(0x7f48800b6ac0 0x7f48801658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2414793717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8048300 0x7f1ce80e7d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ce804f810 tx=0x7f1cec02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c236cb222ddd626 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2014837662 0 0) 0x7f4874001040 con 0x7f48800b66b0 2024-04-25T16:27:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4868003520 con 0x7f48800b66b0 2024-04-25T16:27:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2933475990 0 0) 0x7f4868003520 con 0x7f48800b66b0 2024-04-25T16:27:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.245+0000 7fce5fcd8640 1 -- start start 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 >> v2:172.21.15.138:3300/0 conn(0x7f48800b6ac0 msgr2=0x7f48801658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 --2- 172.21.15.94:0/1353604341 >> v2:172.21.15.138:3300/0 conn(0x7f48800b6ac0 0x7f48801658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 >> v1:172.21.15.138:6789/0 conn(0x7f48800b7200 legacy=0x7f48800b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488006d1d0 con 0x7f48800b66b0 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f4874001770 con 0x7f48800b66b0 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 >> v1:172.21.15.94:6789/0 conn(0x7f48800b66b0 legacy=0x7f48801651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4868003a00 0x7f4868003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdb02bb4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc15e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4868004800 0x7f4868004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4868005270 0x7f486800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f488006d820 con 0x7f4868003a00 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f486800c190 con 0x7f4868004800 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f486800c390 con 0x7f4868005270 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdafbfff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdafc06dc90 0x7fdafc15de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:51640/0 (socket says 172.21.15.94:51640) 2024-04-25T16:27:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdafbfff640 1 -- 172.21.15.94:0/3316872858 learned_addr learned my addr 172.21.15.94:0/3316872858 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fe63ffff640 1 -- 172.21.15.94:0/2227569531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 msgr2=0x7fe6400aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5fcd8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b08b0 0x7fce580cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5fcd8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce580b4d10 0x7fce581ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5fcd8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5815ca70 0x7fce5815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5fcd8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce5815a320 con 0x7fce580b08b0 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5fcd8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce580ce090 con 0x7fce580b4d10 2024-04-25T16:27:22.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5fcd8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce580cfd40 con 0x7fce5815ca70 2024-04-25T16:27:22.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fe63ffff640 1 --2- 172.21.15.94:0/2227569531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 0x7fe6400aff10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe63400b920 tx=0x7fe63402a620 comp rx=0 tx=0).stop 2024-04-25T16:27:22.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5da74640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b08b0 0x7fce580cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5da74640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b08b0 0x7fce580cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55906/0 (socket says 172.21.15.94:55906) 2024-04-25T16:27:22.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5da74640 1 -- 172.21.15.94:0/2925579683 learned_addr learned my addr 172.21.15.94:0/2925579683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f1cf4d38640 1 -- 172.21.15.94:0/2414793717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec037070 con 0x7f1ce8048300 2024-04-25T16:27:22.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f1cf4d38640 1 -- 172.21.15.94:0/2414793717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec02adc0 con 0x7f1ce8048300 2024-04-25T16:27:22.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7f1cf4d38640 1 -- 172.21.15.94:0/2414793717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec032660 con 0x7f1ce8048300 2024-04-25T16:27:22.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fce5d273640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce580b4d10 0x7fce581ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fce5e275640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5815ca70 0x7fce5815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdafc0b03a0 con 0x7fdafc070e80 2024-04-25T16:27:22.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdafc0b0680 con 0x7fdafc06dc90 2024-04-25T16:27:22.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdafc0b0960 con 0x7fdafc15e520 2024-04-25T16:27:22.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdb00950640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdafc070e80 0x7fdafc06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56220/0 (socket says 172.21.15.94:56220) 2024-04-25T16:27:22.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.247+0000 7fdb01151640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f32e605b640 1 Processor -- start 2024-04-25T16:27:22.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487f7fe640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4868005270 0x7f486800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487effd640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4868003a00 0x7f4868003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487e7fc640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4868004800 0x7f4868004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fe63ffff640 1 -- 172.21.15.94:0/2227569531 shutdown_connections 2024-04-25T16:27:22.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fe63ffff640 1 --2- 172.21.15.94:0/2227569531 >> v2:172.21.15.94:3300/0 conn(0x7fe64015f930 0x7fe6400b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fe63ffff640 1 --2- 172.21.15.94:0/2227569531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe64015b0d0 0x7fe6400aff10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fe63ffff640 1 -- 172.21.15.94:0/2227569531 >> 172.21.15.94:0/2227569531 conn(0x7fe640074320 msgr2=0x7fe64006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.246+0000 7f4c19422640 1 -- start start 2024-04-25T16:27:22.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fe63ffff640 1 -- 172.21.15.94:0/2227569531 shutdown_connections 2024-04-25T16:27:22.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f32e605b640 1 -- start start 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fce5d273640 1 -- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5815ca70 msgr2=0x7fce5815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fce5d273640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5815ca70 0x7fce5815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fce5d273640 1 -- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b08b0 msgr2=0x7fce580cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fce5d273640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce580b08b0 0x7fce580cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fce5d273640 1 -- 172.21.15.94:0/2925579683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580cff90 con 0x7fce580b4d10 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fdb01151640 1 -- 172.21.15.94:0/3316872858 >> v1:172.21.15.138:6789/0 conn(0x7fdafc06dc90 legacy=0x7fdafc15de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7fdb01151640 1 -- 172.21.15.94:0/3316872858 >> v1:172.21.15.94:6789/0 conn(0x7fdafc070e80 legacy=0x7fdafc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7fdb01151640 1 -- 172.21.15.94:0/3316872858 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc0b0c40 con 0x7fdafc15e520 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f4c19422640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4c0c034060 0x7f4c0c0e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f4c19422640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c006130 con 0x7f4c0c047db0 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f4c19422640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c04ffa0 con 0x7f4c0c048300 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f4c19422640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4c0c04f690 con 0x7f4c0c034060 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487effd640 1 -- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4868005270 msgr2=0x7f486800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487effd640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4868005270 0x7f486800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487effd640 1 -- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4868004800 msgr2=0x7f4868004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487effd640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4868004800 0x7f4868004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.248+0000 7f487effd640 1 -- 172.21.15.94:0/1353604341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488006d1d0 con 0x7f4868003a00 2024-04-25T16:27:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7fce5d273640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce580b4d10 0x7fce581ef250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce5000ebc0 tx=0x7fce5000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1719be0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17140b2d10 0x7f17140b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1719be0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17140c61a0 0x7f1714159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f4c13fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4c0c047db0 0x7f4c0c033950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56236/0 (socket says 172.21.15.94:56236) 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f4c13fff640 1 -- 172.21.15.94:0/2058466922 learned_addr learned my addr 172.21.15.94:0/2058466922 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f4c137fe640 1 --2- 172.21.15.94:0/2058466922 >> v2:172.21.15.94:3300/0 conn(0x7f4c0c034060 0x7f4c0c0e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1719be0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1714168030 0x7f171409d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1719be0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1714168440 con 0x7f17140c61a0 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1719be0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17140d3570 con 0x7f17140b2d10 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1719be0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f17140d1370 con 0x7f1714168030 2024-04-25T16:27:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f487f7fe640 1 -- 172.21.15.94:0/1353604341 reap_dead start 2024-04-25T16:27:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2414793717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8048300 msgr2=0x7f1ce80e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1cf6d3c640 1 --2- 172.21.15.94:0/2414793717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8048300 0x7f1ce80e7d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ce804f810 tx=0x7f1cec02a0b0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7fe63ffff640 1 -- 172.21.15.94:0/2227569531 wait complete. 2024-04-25T16:27:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1216378663 0 0) 0x7f4c0c006130 con 0x7f4c0c047db0 2024-04-25T16:27:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c04003520 con 0x7f4c0c047db0 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2414793717 shutdown_connections 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1cf6d3c640 1 --2- 172.21.15.94:0/2414793717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8048300 0x7f1ce80e7d50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2414793717 >> 172.21.15.94:0/2414793717 conn(0x7f1ce8054cf0 msgr2=0x7f1ce8055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2414793717 shutdown_connections 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2414793717 wait complete. 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1564597327 0 0) 0x7f4c04003520 con 0x7f4c0c047db0 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 >> v2:172.21.15.94:3300/0 conn(0x7f4c0c034060 msgr2=0x7f4c0c0e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 --2- 172.21.15.94:0/2058466922 >> v2:172.21.15.94:3300/0 conn(0x7f4c0c034060 0x7f4c0c0e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f17193df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1714168030 0x7f171409d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f17193df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1714168030 0x7f171409d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:59384/0 (socket says 172.21.15.94:59384) 2024-04-25T16:27:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f17193df640 1 -- 172.21.15.94:0/2415096774 learned_addr learned my addr 172.21.15.94:0/2415096774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f1713fff640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17140c61a0 0x7f1714159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e605b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 0x7f32d804bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f487effd640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4868003a00 0x7f4868003e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4880073460 tx=0x7f4868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3042f9ee3941bd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 >> v1:172.21.15.138:6789/0 conn(0x7f4c0c048300 legacy=0x7f4c0c0065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c050280 con 0x7f4c0c047db0 2024-04-25T16:27:22.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e605b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d8054560 0x7f32d80d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f1718bde640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17140b2d10 0x7f17140b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e605b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d80e04d0 0x7f32d80e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7fdb01151640 1 --2- 172.21.15.94:0/3316872858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc15e930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009b40 tx=0x7fdaf402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c1d1c79a600f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e605b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8007d60 con 0x7f32d80e04d0 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e605b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32d80e08e0 con 0x7f32d80092a0 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e605b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32d8055480 con 0x7f32d8054560 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e5059640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 0x7f32d804bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e4858640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d8054560 0x7f32d80d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.249+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50018020 con 0x7fce580b4d10 2024-04-25T16:27:22.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5000cda0 con 0x7fce580b4d10 2024-04-25T16:27:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500146c0 con 0x7fce580b4d10 2024-04-25T16:27:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce580d0270 con 0x7fce580b4d10 2024-04-25T16:27:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce581489e0 con 0x7fce580b4d10 2024-04-25T16:27:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f4c14076bb0 con 0x7f4c0c047db0 2024-04-25T16:27:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870001440 con 0x7f4868003a00 2024-04-25T16:27:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f17193df640 1 -- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17140b2d10 msgr2=0x7f17140b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f17193df640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17140b2d10 0x7f17140b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f487dffb640 1 -- 172.21.15.94:0/1353604341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487002e070 con 0x7f4868003a00 2024-04-25T16:27:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f17193df640 1 -- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17140c61a0 msgr2=0x7f1714159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f17193df640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17140c61a0 0x7f1714159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f17193df640 1 -- 172.21.15.94:0/2415096774 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1714157430 con 0x7f1714168030 2024-04-25T16:27:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e4858640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d8054560 0x7f32d80d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:59398/0 (socket says 172.21.15.94:59398) 2024-04-25T16:27:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e4858640 1 -- 172.21.15.94:0/4022568133 learned_addr learned my addr 172.21.15.94:0/4022568133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f32e585a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d80e04d0 0x7f32d80e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f1718bde640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17140b2d10 0x7f17140b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f1713fff640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17140c61a0 0x7f1714159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7fdafb7fe640 1 -- 172.21.15.94:0/3316872858 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4038070 con 0x7fdafc15e520 2024-04-25T16:27:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7fdafb7fe640 1 -- 172.21.15.94:0/3316872858 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf4007740 con 0x7fdafc15e520 2024-04-25T16:27:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7fdafb7fe640 1 -- 172.21.15.94:0/3316872858 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4007af0 con 0x7fdafc15e520 2024-04-25T16:27:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f17193df640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1714168030 0x7f171409d4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f170400d4e0 tx=0x7f170400d9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f32e585a640 1 -- 172.21.15.94:0/4022568133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d8054560 msgr2=0x7f32d80d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.250+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 >> v1:172.21.15.94:6789/0 conn(0x7f4c0c047db0 legacy=0x7f4c0c033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f32e585a640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d8054560 0x7f32d80d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f32e585a640 1 -- 172.21.15.94:0/4022568133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 msgr2=0x7f32d804bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f32e585a640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d80092a0 0x7f32d804bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f32e585a640 1 -- 172.21.15.94:0/4022568133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d80e10e0 con 0x7f32d80e04d0 2024-04-25T16:27:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f32e4858640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d8054560 0x7f32d80d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040125f0 con 0x7f1714168030 2024-04-25T16:27:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.251+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce58076bb0 con 0x7fce580b4d10 2024-04-25T16:27:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704019070 con 0x7f1714168030 2024-04-25T16:27:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040129d0 con 0x7f1714168030 2024-04-25T16:27:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f32e585a640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d80e04d0 0x7f32d80e2e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32e006b3e0 tx=0x7f32e006ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1714157710 con 0x7f1714168030 2024-04-25T16:27:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f48852f9640 1 -- 172.21.15.94:0/1353604341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4868003a00 msgr2=0x7f4868003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f48852f9640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4868003a00 0x7f4868003e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4880073460 tx=0x7f4868005830 comp rx=0 tx=0).stop 2024-04-25T16:27:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c12ffd640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141579f0 con 0x7f1714168030 2024-04-25T16:27:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7fe63ffff640 1 Processor -- start 2024-04-25T16:27:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f48852f9640 1 -- 172.21.15.94:0/1353604341 shutdown_connections 2024-04-25T16:27:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f48852f9640 1 --2- 172.21.15.94:0/1353604341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4868003a00 0x7f4868003e70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f48852f9640 1 -- 172.21.15.94:0/1353604341 >> 172.21.15.94:0/1353604341 conn(0x7f48800711c0 msgr2=0x7f48800715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7fe63ffff640 1 -- start start 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c12ffd640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c04004800 0x7f4c04004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c12ffd640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04005150 0x7f4c0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c04f690 con 0x7f4c040039b0 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0400c0c0 con 0x7f4c04005150 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0400c2c0 con 0x7f4c04004800 2024-04-25T16:27:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f1cf6d3c640 1 Processor -- start 2024-04-25T16:27:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c137fe640 1 --2- 172.21.15.94:0/2058466922 >> v2:172.21.15.94:3300/0 conn(0x7f4c0c034060 0x7f4c0c0e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f4c18c21640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04005150 0x7f4c0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 msgr2=0x7fdafc15e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7fdb02bb4640 1 --2- 172.21.15.94:0/3316872858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc15e930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009b40 tx=0x7fdaf402b930 comp rx=0 tx=0).stop 2024-04-25T16:27:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c13fff640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 shutdown_connections 2024-04-25T16:27:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7fdb02bb4640 1 --2- 172.21.15.94:0/3316872858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc15e930 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 >> 172.21.15.94:0/3316872858 conn(0x7fdafc073500 msgr2=0x7fdafc073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00be020 con 0x7f32d80e04d0 2024-04-25T16:27:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e00ad990 con 0x7f32d80e04d0 2024-04-25T16:27:22.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b3040 con 0x7f32d80e04d0 2024-04-25T16:27:22.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f48852f9640 1 -- 172.21.15.94:0/1353604341 shutdown_connections 2024-04-25T16:27:22.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c13fff640 1 -- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c04004800 msgr2=0x7f4c04004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c13fff640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c04004800 0x7f4c04004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c13fff640 1 -- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04005150 msgr2=0x7f4c0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:22.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 shutdown_connections 2024-04-25T16:27:22.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7fdb02bb4640 1 -- 172.21.15.94:0/3316872858 wait complete. 2024-04-25T16:27:22.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c13fff640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04005150 0x7f4c0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c13fff640 1 -- 172.21.15.94:0/2058466922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c04003520 con 0x7f4c040039b0 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f4c18c21640 1 -- 172.21.15.94:0/2058466922 reap_dead start 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f4c13fff640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c04003de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c140519b0 tx=0x7f4c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a529dc5129d9d905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1406a3b0 con 0x7f4c040039b0 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1704012b70 con 0x7f1714168030 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1714076bb0 con 0x7f1714168030 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f32e605b640 1 -- 172.21.15.94:0/4022568133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d80e13c0 con 0x7f32d80e04d0 2024-04-25T16:27:22.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.253+0000 7f32e605b640 1 -- 172.21.15.94:0/4022568133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d804a4d0 con 0x7f32d80e04d0 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32d8005570 con 0x7f32d80e04d0 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce50005d10 con 0x7fce580b4d10 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fce4e7fc640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fce3c074580 0x7fce3c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce50096b70 con 0x7fce580b4d10 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6400cdf70 0x7fe6400cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe64015b0d0 0x7fe64014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe64015f930 0x7fe640154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63ffff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe640161ac0 con 0x7fe64015b0d0 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63ffff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe640161940 con 0x7fe64015f930 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63ffff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe640161c40 con 0x7fe6400cdf70 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe64015f930 0x7fe640154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe64015f930 0x7fe640154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41284/0 (socket says 172.21.15.94:41284) 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 -- 172.21.15.94:0/1871393515 learned_addr learned my addr 172.21.15.94:0/1871393515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63effd640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6400cdf70 0x7fe6400cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63e7fc640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe64015b0d0 0x7fe64014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f17117fa640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1700074580 0x7f1700076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fce4e7fc640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 0x7fce3c0801e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 --> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce50014e50 con 0x7fce3c07dd50 2024-04-25T16:27:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce50096f60 con 0x7fce580b4d10 2024-04-25T16:27:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c140ac070 con 0x7f4c040039b0 2024-04-25T16:27:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f4c12ffd640 1 -- 172.21.15.94:0/2058466922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1406e270 con 0x7f4c040039b0 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1704095780 con 0x7f1714168030 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 -- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6400cdf70 msgr2=0x7fe6400cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6400cdf70 0x7fe6400cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 -- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe64015b0d0 msgr2=0x7fe64014f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe64015b0d0 0x7fe64014f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fe63f7fe640 1 -- 172.21.15.94:0/1871393515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6401559d0 con 0x7fe64015f930 2024-04-25T16:27:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7fce5da74640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fce3c074580 0x7fce3c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f17117fa640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 0x7f1700080150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1718bde640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1700074580 0x7f1700076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.255+0000 7f48852f9640 1 -- 172.21.15.94:0/1353604341 wait complete. 2024-04-25T16:27:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7fe63f7fe640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe64015f930 0x7fe640154950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe638007220 tx=0x7fe63800f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 --> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1700080820 con 0x7f170007dcc0 2024-04-25T16:27:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f170409c050 con 0x7f1714168030 2024-04-25T16:27:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638003da0 con 0x7fe64015f930 2024-04-25T16:27:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1713fff640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 0x7f1700080150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63801c070 con 0x7fe64015f930 2024-04-25T16:27:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6400cf190 con 0x7fe64015f930 2024-04-25T16:27:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64015a890 con 0x7fe64015f930 2024-04-25T16:27:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7fce5e275640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 0x7fce3c0801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638004250 con 0x7fe64015f930 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.252+0000 7f1cf6d3c640 1 -- start start 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.254+0000 7f1cf6d3c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8047db0 0x7f1ce80d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf5d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8047db0 0x7f1ce80d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf5d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8047db0 0x7f1ce80d3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55984/0 (socket says 172.21.15.94:55984) 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf5d3a640 1 -- 172.21.15.94:0/2173841141 learned_addr learned my addr 172.21.15.94:0/2173841141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf6d3c640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ce8048300 0x7f1ce80d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf6d3c640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ce8035b20 0x7f1ce8035f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ce8035b20 0x7f1ce8035f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf5539640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ce8048300 0x7f1ce80d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80509c0 con 0x7f1ce8047db0 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8050840 con 0x7f1ce8048300 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.256+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce8006cf0 con 0x7f1ce8035b20 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5d3a640 1 -- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ce8035b20 msgr2=0x7f1ce8035f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5d3a640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ce8035b20 0x7f1ce8035f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5d3a640 1 -- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ce8048300 msgr2=0x7f1ce80d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5d3a640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ce8048300 0x7f1ce80d6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5d3a640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce8036470 con 0x7f1ce8047db0 2024-04-25T16:27:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7fdb02bb4640 1 Processor -- start 2024-04-25T16:27:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5539640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ce8048300 0x7f1ce80d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f4c19422640 1 -- 172.21.15.94:0/2058466922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 msgr2=0x7f4c04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f4c19422640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c04003de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c140519b0 tx=0x7f4c04005830 comp rx=0 tx=0).stop 2024-04-25T16:27:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f32e00c1020 con 0x7f32d80e04d0 2024-04-25T16:27:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1cf5d3a640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8047db0 0x7f1ce80d3da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0077b00 tx=0x7f1cf00758c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7f1718bde640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1700074580 0x7f1700076a90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1708034000 tx=0x7f170802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7fce5da74640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fce3c074580 0x7fce3c076a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce5402a440 tx=0x7fce5402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe620001030 con 0x7fe64015f930 2024-04-25T16:27:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.257+0000 7fdb02bb4640 1 -- start start 2024-04-25T16:27:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f4c19422640 1 -- 172.21.15.94:0/2058466922 shutdown_connections 2024-04-25T16:27:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f4c19422640 1 --2- 172.21.15.94:0/2058466922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c04003de0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f4c19422640 1 -- 172.21.15.94:0/2058466922 >> 172.21.15.94:0/2058466922 conn(0x7f4c0c054cf0 msgr2=0x7f4c0c055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf006e110 con 0x7f1ce8047db0 2024-04-25T16:27:22.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f48852f9640 1 Processor -- start 2024-04-25T16:27:22.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f32d5ffb640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f32c4074530 0x7f32c4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7fce5e275640 1 -- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 msgr2=0x7fce3c0801e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7fce5e275640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 0x7fce3c0801e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f48852f9640 1 -- start start 2024-04-25T16:27:22.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf006e2b0 con 0x7f1ce8047db0 2024-04-25T16:27:22.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce80d4550 con 0x7f1ce8047db0 2024-04-25T16:27:22.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf006ece0 con 0x7f1ce8047db0 2024-04-25T16:27:22.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f1713fff640 1 -- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 msgr2=0x7f1700080150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f1713fff640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 0x7f1700080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f32e5059640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f32c4074530 0x7f32c4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f4c19422640 1 -- 172.21.15.94:0/2058466922 shutdown_connections 2024-04-25T16:27:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe638016070 con 0x7fe64015f930 2024-04-25T16:27:22.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7fe6277fe640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe62c074580 0x7fe62c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdb02bb4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc06dc90 0x7fdafc0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f1713fff640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 0x7f1700080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f1700080820 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.258+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f32e0132d40 con 0x7f32d80e04d0 2024-04-25T16:27:22.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7f32d5ffb640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 0x7f32c4080110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 --> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32c40807e0 con 0x7f32c407dc80 2024-04-25T16:27:22.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7f32e4858640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 0x7f32c4080110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe638099810 con 0x7fe64015f930 2024-04-25T16:27:22.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fe6277fe640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 0x7fe62c080160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f32e4858640 1 -- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 msgr2=0x7f32c4080110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f32e4858640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 0x7f32c4080110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 --> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe638017e50 con 0x7fe62c07dcd0 2024-04-25T16:27:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe63805f4b0 con 0x7fe64015f930 2024-04-25T16:27:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7f4c19422640 1 -- 172.21.15.94:0/2058466922 wait complete. 2024-04-25T16:27:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32e00f9270 con 0x7f32d80e04d0 2024-04-25T16:27:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdb02bb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdafc070e80 0x7fdafc09f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdb00950640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc06dc90 0x7fdafc0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f32e5059640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f32c4074530 0x7f32c4076a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f32dc000f60 tx=0x7f32dc001650 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdb00950640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc06dc90 0x7fdafc0d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41316/0 (socket says 172.21.15.94:41316) 2024-04-25T16:27:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdafc070e80 0x7fdafc09f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdafc070e80 0x7fdafc09f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56014/0 (socket says 172.21.15.94:56014) 2024-04-25T16:27:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f48852f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f488015f650 0x7f488015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f48852f9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48801441b0 0x7f48801445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f48852f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4880144b00 0x7f48801fe200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f48852f9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48800cfa10 con 0x7f4880144b00 2024-04-25T16:27:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f48852f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f488014e8a0 con 0x7f488015f650 2024-04-25T16:27:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f48852f9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48800cfb90 con 0x7f48801441b0 2024-04-25T16:27:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fce5e275640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 0x7fce3c0801e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fce50014e50 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7fdb00950640 1 -- 172.21.15.94:0/461646413 learned_addr learned my addr 172.21.15.94:0/461646413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.259+0000 7f1cf6d3c640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce817dd50 con 0x7f1ce8047db0 2024-04-25T16:27:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.260+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ccc001030 con 0x7f1ce8047db0 2024-04-25T16:27:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdb02bb4640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc1fe190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc15eff0 con 0x7fdafc070e80 2024-04-25T16:27:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc15ee70 con 0x7fdafc06dc90 2024-04-25T16:27:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdafc071290 con 0x7fdafc15e520 2024-04-25T16:27:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fe63effd640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe62c074580 0x7fe62c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487e7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48801441b0 0x7f48801445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487e7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48801441b0 0x7f48801445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:59424/0 (socket says 172.21.15.94:59424) 2024-04-25T16:27:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487e7fc640 1 -- 172.21.15.94:0/3066336106 learned_addr learned my addr 172.21.15.94:0/3066336106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f32e4858640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 0x7f32c4080110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f32c40807e0 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1cf006c570 con 0x7f1ce8047db0 2024-04-25T16:27:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487f7fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4880144b00 0x7f48801fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487effd640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f488015f650 0x7f488015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fe63e7fc640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 0x7fe62c080160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f4c19422640 1 Processor -- start 2024-04-25T16:27:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487e7fc640 1 -- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f488015f650 msgr2=0x7f488015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487e7fc640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f488015f650 0x7f488015ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7f487e7fc640 1 -- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4880144b00 msgr2=0x7f48801fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdb01151640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc1fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fe63effd640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe62c074580 0x7fe62c076a90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe634034000 tx=0x7fe63402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f1ce67fc640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1cd40744b0 0x7f1cd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f4c19422640 1 -- start start 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f487e7fc640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4880144b00 0x7f48801fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f487e7fc640 1 -- 172.21.15.94:0/3066336106 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4880143d80 con 0x7f48801441b0 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdafbfff640 1 -- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 msgr2=0x7fdafc1fe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdafbfff640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdafc15e520 0x7fdafc1fe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdafbfff640 1 -- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc06dc90 msgr2=0x7fdafc0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdafbfff640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc06dc90 0x7fdafc0d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdafbfff640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc153d20 con 0x7fdafc070e80 2024-04-25T16:27:22.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.261+0000 7fdb00950640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc06dc90 0x7fdafc0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:22.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1cf0111f80 con 0x7f1ce8047db0 2024-04-25T16:27:22.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f487e7fc640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48801441b0 0x7f48801445c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f487400f900 tx=0x7f487400fdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fdafbfff640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdafc070e80 0x7fdafc09f170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c4d0 tx=0x7fdaec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f1ce67fc640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 0x7f1cd4080090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cf0074c20 con 0x7f1cd407dc00 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cf0114560 con 0x7f1ce8047db0 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f1cf5539640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1cd40744b0 0x7f1cd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec018070 con 0x7fdafc070e80 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec0074e0 con 0x7fdafc070e80 2024-04-25T16:27:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48740052e0 con 0x7f48801441b0 2024-04-25T16:27:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0049e0 con 0x7fdafc070e80 2024-04-25T16:27:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc154000 con 0x7fdafc070e80 2024-04-25T16:27:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4874018070 con 0x7f48801441b0 2024-04-25T16:27:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400d9d0 con 0x7f48801441b0 2024-04-25T16:27:22.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fe63e7fc640 1 -- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 msgr2=0x7fe62c080160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fe63e7fc640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 0x7fe62c080160 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4880143200 con 0x7f48801441b0 2024-04-25T16:27:22.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1545c0 con 0x7fdafc070e80 2024-04-25T16:27:22.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.262+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801437c0 con 0x7f48801441b0 2024-04-25T16:27:22.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.263+0000 7fe63e7fc640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 0x7fe62c080160 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fe638017e50 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.263+0000 7f1cf5539640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1cd40744b0 0x7f1cd40769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0000f60 tx=0x7f1ce0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.263+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdadc001030 con 0x7fdafc070e80 2024-04-25T16:27:22.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.263+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 0x7f1cd4080090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f488015f470 con 0x7f48801441b0 2024-04-25T16:27:22.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c19422640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c0c044990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c0c044990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c0c044990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56030/0 (socket says 172.21.15.94:56030) 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c19422640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c0c0d90d0 0x7f4c0c0d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c13fff640 1 -- 172.21.15.94:0/1103741722 learned_addr learned my addr 172.21.15.94:0/1103741722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c19422640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c0c0d94e0 0x7f4c0c0da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f1cf653b640 1 -- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 msgr2=0x7f1cd4080090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 0x7f1cd4080090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c137fe640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c0c0d90d0 0x7f4c0c0d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0509c0 con 0x7f4c040039b0 2024-04-25T16:27:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c18c21640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c0c0d94e0 0x7f4c0c0da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c050840 con 0x7f4c0c0d90d0 2024-04-25T16:27:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c0e7d50 con 0x7f4c0c0d94e0 2024-04-25T16:27:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.264+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdaec004b80 con 0x7fdafc070e80 2024-04-25T16:27:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f4c13fff640 1 -- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c0c0d94e0 msgr2=0x7f4c0c0da1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 0x7f1cd4080090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f1cf0074c20 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f4c13fff640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c0c0d94e0 0x7f4c0c0da1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7fdaf97fa640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdae80744e0 0x7fdae80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f4c13fff640 1 -- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c0c0d90d0 msgr2=0x7f4c0c0d9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:22.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f4c13fff640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c0c0d90d0 0x7f4c0c0d9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:22.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f4c13fff640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c045420 con 0x7f4c040039b0 2024-04-25T16:27:22.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f487400db70 con 0x7f48801441b0 2024-04-25T16:27:22.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdaec094730 con 0x7fdafc070e80 2024-04-25T16:27:22.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7fdaf97fa640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 0x7fdae80800b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7f4c13fff640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c0c044990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c1402e0e0 tx=0x7f4c14076330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdae8080780 con 0x7fdae807dc20 2024-04-25T16:27:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.265+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdaec09b050 con 0x7fdafc070e80 2024-04-25T16:27:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7fdb01151640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 0x7fdae80800b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7fdb00950640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdae80744e0 0x7fdae80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1406e070 con 0x7f4c040039b0 2024-04-25T16:27:22.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f48677fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f486c0744e0 0x7f486c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1406e210 con 0x7f4c040039b0 2024-04-25T16:27:22.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1406c770 con 0x7f4c040039b0 2024-04-25T16:27:22.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4874095e70 con 0x7f48801441b0 2024-04-25T16:27:22.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f487effd640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f486c0744e0 0x7f486c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c045700 con 0x7f4c040039b0 2024-04-25T16:27:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c045cc0 con 0x7f4c040039b0 2024-04-25T16:27:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7fdb00950640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdae80744e0 0x7fdae80769f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00012a0 tx=0x7fdaf0001230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f48677fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 0x7f486c0800c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 --> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f486c080790 con 0x7f486c07dc30 2024-04-25T16:27:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4874096300 con 0x7f48801441b0 2024-04-25T16:27:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7f487f7fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 0x7f486c0800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7fdb01151640 1 -- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 msgr2=0x7fdae80800b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.266+0000 7fdb01151640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 0x7fdae80800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.267+0000 7f487f7fe640 1 -- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 msgr2=0x7f486c0800c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.267+0000 7f487f7fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 0x7f486c0800c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.267+0000 7f487effd640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f486c0744e0 0x7f486c0769f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4870032000 tx=0x7f4870007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.267+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c0c005570 con 0x7f4c040039b0 2024-04-25T16:27:22.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.267+0000 7fdb01151640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 0x7fdae80800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fdae8080780 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.268+0000 7f487f7fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 0x7f486c0800c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f486c080790 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.268+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4c140ba370 con 0x7f4c040039b0 2024-04-25T16:27:22.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.268+0000 7f4c10ff9640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4bf80744b0 0x7f4bf80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.268+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4c14131f40 con 0x7f4c040039b0 2024-04-25T16:27:22.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.268+0000 7f4c137fe640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4bf80744b0 0x7f4bf80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.268+0000 7f4c10ff9640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 0x7f4bf8080080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.269+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bf8080750 con 0x7f4bf807dbf0 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.269+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c14139050 con 0x7f4c040039b0 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.269+0000 7f4c18c21640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 0x7f4bf8080080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.269+0000 7f4c137fe640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4bf80744b0 0x7f4bf80769c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0e5cb0 tx=0x7f4c00004730 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.269+0000 7f4c18c21640 1 -- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 msgr2=0x7f4bf8080080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.269+0000 7f4c18c21640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 0x7f4bf8080080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:22.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.270+0000 7f4c18c21640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 0x7f4bf8080080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f4bf8080750 command_reply(tid 1: 0 ) 2024-04-25T16:27:22.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.704+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== osd.2 v2:172.21.15.94:6824/2599289524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1700080820 con 0x7f170007dcc0 2024-04-25T16:27:22.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.705+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== osd.3 v2:172.21.15.94:6808/1336839764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32c40807e0 con 0x7f32c407dc80 2024-04-25T16:27:22.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.705+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== osd.4 v2:172.21.15.138:6816/3435025595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce50014e50 con 0x7fce3c07dd50 2024-04-25T16:27:22.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.707+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== osd.7 v2:172.21.15.138:6808/300369448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe638017e50 con 0x7fe62c07dcd0 2024-04-25T16:27:22.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.709+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== osd.6 v2:172.21.15.138:6800/62635939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cf0074c20 con 0x7f1cd407dc00 2024-04-25T16:27:22.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.711+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== osd.0 v2:172.21.15.94:6809/2433651201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdae8080780 con 0x7fdae807dc20 2024-04-25T16:27:22.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.712+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== osd.5 v2:172.21.15.138:6824/4211896878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f486c080790 con 0x7f486c07dc30 2024-04-25T16:27:22.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.714+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== osd.1 v2:172.21.15.94:6800/816278269 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bf8080750 con 0x7f4bf807dbf0 2024-04-25T16:27:22.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.730+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 --> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32d80e0e30 con 0x7f32c407dc80 2024-04-25T16:27:22.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.731+0000 7f32e4858640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 0x7f32c4080110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f32d80e0e30 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.732+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 --> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe620002dd0 con 0x7fe62c07dcd0 2024-04-25T16:27:22.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.733+0000 7fe63e7fc640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 0x7fe62c080160 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fe620002dd0 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.736+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 --> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17140b5350 con 0x7f170007dcc0 2024-04-25T16:27:22.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.736+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 --> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdadc002dd0 con 0x7fdae807dc20 2024-04-25T16:27:22.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.736+0000 7f1713fff640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 0x7f1700080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f17140b5350 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.737+0000 7fdb01151640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 0x7fdae80800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fdadc002dd0 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.737+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 --> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce58076bb0 con 0x7fce3c07dd50 2024-04-25T16:27:22.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.738+0000 7fce5e275640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 0x7fce3c0801e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fce58076bb0 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.739+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 --> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4c0c0e5ce0 con 0x7f4bf807dbf0 2024-04-25T16:27:22.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.739+0000 7f4c18c21640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 0x7f4bf8080080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f4c0c0e5ce0 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.739+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 --> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1ccc002dd0 con 0x7f1cd407dc00 2024-04-25T16:27:22.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.740+0000 7f1cf653b640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 0x7f1cd4080090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f1ccc002dd0 command_reply(tid 2: 0 ) 2024-04-25T16:27:22.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.741+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 --> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f488006a180 con 0x7f486c07dc30 2024-04-25T16:27:22.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:22.742+0000 7f487f7fe640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 0x7f486c0800c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f488006a180 command_reply(tid 2: 0 ) 2024-04-25T16:27:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32d5ffb640 1 -- 172.21.15.94:0/4022568133 <== osd.3 v2:172.21.15.94:6808/1336839764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f32d80e0e30 con 0x7f32c407dc80 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 msgr2=0x7f32c4080110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f32c407dc80 0x7f32c4080110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f32c4074530 msgr2=0x7f32c4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f32c4074530 0x7f32c4076a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f32dc000f60 tx=0x7f32dc001650 comp rx=0 tx=0).stop 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d80e04d0 msgr2=0x7f32d80e2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 --2- 172.21.15.94:0/4022568133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d80e04d0 0x7f32d80e2e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32e006b3e0 tx=0x7f32e006ada0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32e585a640 1 -- 172.21.15.94:0/4022568133 reap_dead start 2024-04-25T16:27:26.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 shutdown_connections 2024-04-25T16:27:26.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 >> 172.21.15.94:0/4022568133 conn(0x7f32d8055e00 msgr2=0x7f32d8054bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 shutdown_connections 2024-04-25T16:27:26.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.192+0000 7f32c37fe640 1 -- 172.21.15.94:0/4022568133 wait complete. 2024-04-25T16:27:26.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe6277fe640 1 -- 172.21.15.94:0/1871393515 <== osd.7 v2:172.21.15.138:6808/300369448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe620002dd0 con 0x7fe62c07dcd0 2024-04-25T16:27:26.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 msgr2=0x7fe62c080160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7fe62c07dcd0 0x7fe62c080160 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe62c074580 msgr2=0x7fe62c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe62c074580 0x7fe62c076a90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe634034000 tx=0x7fe63402f000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe64015f930 msgr2=0x7fe640154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 --2- 172.21.15.94:0/1871393515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe64015f930 0x7fe640154950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe638007220 tx=0x7fe63800f8f0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63f7fe640 1 -- 172.21.15.94:0/1871393515 reap_dead start 2024-04-25T16:27:26.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 shutdown_connections 2024-04-25T16:27:26.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 >> 172.21.15.94:0/1871393515 conn(0x7fe640074320 msgr2=0x7fe640160f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 shutdown_connections 2024-04-25T16:27:26.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.193+0000 7fe63ffff640 1 -- 172.21.15.94:0/1871393515 wait complete. 2024-04-25T16:27:26.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f17117fa640 1 -- 172.21.15.94:0/2415096774 <== osd.2 v2:172.21.15.94:6824/2599289524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f17140b5350 con 0x7f170007dcc0 2024-04-25T16:27:26.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7fdaf97fa640 1 -- 172.21.15.94:0/461646413 <== osd.0 v2:172.21.15.94:6809/2433651201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdadc002dd0 con 0x7fdae807dc20 2024-04-25T16:27:26.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 msgr2=0x7f1700080150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f1719be0640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f170007dcc0 0x7f1700080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1700074580 msgr2=0x7f1700076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f1719be0640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1700074580 0x7f1700076a90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1708034000 tx=0x7f170802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1714168030 msgr2=0x7f171409d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f1719be0640 1 --2- 172.21.15.94:0/2415096774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1714168030 0x7f171409d4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f170400d4e0 tx=0x7f170400d9a0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7f17193df640 1 -- 172.21.15.94:0/2415096774 reap_dead start 2024-04-25T16:27:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 msgr2=0x7fdae80800b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7fdb02bb4640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fdae807dc20 0x7fdae80800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdae80744e0 msgr2=0x7fdae80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7fdb02bb4640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdae80744e0 0x7fdae80769f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00012a0 tx=0x7fdaf0001230 comp rx=0 tx=0).stop 2024-04-25T16:27:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 shutdown_connections 2024-04-25T16:27:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.197+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdafc070e80 msgr2=0x7fdafc09f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fdb02bb4640 1 --2- 172.21.15.94:0/461646413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdafc070e80 0x7fdafc09f170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c4d0 tx=0x7fdaec00c990 comp rx=0 tx=0).stop 2024-04-25T16:27:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fdb01151640 1 -- 172.21.15.94:0/461646413 reap_dead start 2024-04-25T16:27:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fce4e7fc640 1 -- 172.21.15.94:0/2925579683 <== osd.4 v2:172.21.15.138:6816/3435025595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce58076bb0 con 0x7fce3c07dd50 2024-04-25T16:27:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 shutdown_connections 2024-04-25T16:27:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 >> 172.21.15.94:0/461646413 conn(0x7fdafc073500 msgr2=0x7fdafc073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 >> 172.21.15.94:0/2415096774 conn(0x7f171409b9b0 msgr2=0x7f17140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 shutdown_connections 2024-04-25T16:27:26.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 shutdown_connections 2024-04-25T16:27:26.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7f1719be0640 1 -- 172.21.15.94:0/2415096774 wait complete. 2024-04-25T16:27:26.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fdb02bb4640 1 -- 172.21.15.94:0/461646413 wait complete. 2024-04-25T16:27:26.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 msgr2=0x7fce3c0801e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.198+0000 7fce5fcd8640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fce3c07dd50 0x7fce3c0801e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7f4c10ff9640 1 -- 172.21.15.94:0/1103741722 <== osd.1 v2:172.21.15.94:6800/816278269 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4c0c0e5ce0 con 0x7f4bf807dbf0 2024-04-25T16:27:26.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fce3c074580 msgr2=0x7fce3c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fce3c074580 0x7fce3c076a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce5402a440 tx=0x7fce5402e000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce580b4d10 msgr2=0x7fce581ef250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 --2- 172.21.15.94:0/2925579683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce580b4d10 0x7fce581ef250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce5000ebc0 tx=0x7fce5000c490 comp rx=0 tx=0).stop 2024-04-25T16:27:26.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5e275640 1 -- 172.21.15.94:0/2925579683 reap_dead start 2024-04-25T16:27:26.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 shutdown_connections 2024-04-25T16:27:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 >> 172.21.15.94:0/2925579683 conn(0x7fce580733f0 msgr2=0x7fce580b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 shutdown_connections 2024-04-25T16:27:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7fce5fcd8640 1 -- 172.21.15.94:0/2925579683 wait complete. 2024-04-25T16:27:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 msgr2=0x7f4bf8080080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7f4c19422640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f4bf807dbf0 0x7f4bf8080080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f1ce67fc640 1 -- 172.21.15.94:0/2173841141 <== osd.6 v2:172.21.15.138:6800/62635939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1ccc002dd0 con 0x7f1cd407dc00 2024-04-25T16:27:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4bf80744b0 msgr2=0x7f4bf80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.199+0000 7f4c19422640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4bf80744b0 0x7f4bf80769c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0e5cb0 tx=0x7f4c00004730 comp rx=0 tx=0).stop 2024-04-25T16:27:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 msgr2=0x7f4c0c044990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f4c19422640 1 --2- 172.21.15.94:0/1103741722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c040039b0 0x7f4c0c044990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c1402e0e0 tx=0x7f4c14076330 comp rx=0 tx=0).stop 2024-04-25T16:27:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f4c18c21640 1 -- 172.21.15.94:0/1103741722 reap_dead start 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 shutdown_connections 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 >> 172.21.15.94:0/1103741722 conn(0x7f4c0c054cf0 msgr2=0x7f4c0c038830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 msgr2=0x7f1cd4080090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f1cd3fff640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f1cd407dc00 0x7f1cd4080090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1cd40744b0 msgr2=0x7f1cd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.200+0000 7f1cd3fff640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1cd40744b0 0x7f1cd40769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0000f60 tx=0x7f1ce0009040 comp rx=0 tx=0).stop 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8047db0 msgr2=0x7f1ce80d3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cd3fff640 1 --2- 172.21.15.94:0/2173841141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8047db0 0x7f1ce80d3da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0077b00 tx=0x7f1cf00758c0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 shutdown_connections 2024-04-25T16:27:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cf653b640 1 -- 172.21.15.94:0/2173841141 reap_dead start 2024-04-25T16:27:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 shutdown_connections 2024-04-25T16:27:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 >> 172.21.15.94:0/2173841141 conn(0x7f1ce8054cf0 msgr2=0x7f1ce80e63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f4c19422640 1 -- 172.21.15.94:0/1103741722 wait complete. 2024-04-25T16:27:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 shutdown_connections 2024-04-25T16:27:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.201+0000 7f1cd3fff640 1 -- 172.21.15.94:0/2173841141 wait complete. 2024-04-25T16:27:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.203+0000 7f48677fe640 1 -- 172.21.15.94:0/3066336106 <== osd.5 v2:172.21.15.138:6824/4211896878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f488006a180 con 0x7f486c07dc30 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.203+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 msgr2=0x7f486c0800c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.203+0000 7f48852f9640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f486c07dc30 0x7f486c0800c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f486c0744e0 msgr2=0x7f486c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f486c0744e0 0x7f486c0769f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4870032000 tx=0x7f4870007000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48801441b0 msgr2=0x7f48801445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 --2- 172.21.15.94:0/3066336106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48801441b0 0x7f48801445c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f487400f900 tx=0x7f487400fdc0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f487f7fe640 1 -- 172.21.15.94:0/3066336106 reap_dead start 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 shutdown_connections 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 >> 172.21.15.94:0/3066336106 conn(0x7f48800711c0 msgr2=0x7f48800d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 shutdown_connections 2024-04-25T16:27:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.204+0000 7f48852f9640 1 -- 172.21.15.94:0/3066336106 wait complete. 2024-04-25T16:27:26.215 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:27:26.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T16:27:26.217 INFO:teuthology.orchestra.run.smithi094.stdout:60129542156 2024-04-25T16:27:26.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T16:27:26.225 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:27:26.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T16:27:26.227 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:27:26.227 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:27:26.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T16:27:26.228 INFO:teuthology.orchestra.run.smithi094.stdout:60129542155 2024-04-25T16:27:26.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T16:27:26.229 INFO:teuthology.orchestra.run.smithi094.stdout:60129542155 2024-04-25T16:27:26.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T16:27:26.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T16:27:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:60129542155 2024-04-25T16:27:26.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T16:27:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.378+0000 7fa51ae20640 1 Processor -- start 2024-04-25T16:27:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.379+0000 7faf5c7ce640 1 Processor -- start 2024-04-25T16:27:26.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.378+0000 7fa51ae20640 1 -- start start 2024-04-25T16:27:26.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.381+0000 7fa51ae20640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa5140b6cb0 0x7fa5140b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.381+0000 7fa51ae20640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5141610e0 con 0x7fa514070c10 2024-04-25T16:27:26.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.381+0000 7fa51ae20640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa514160560 con 0x7fa51406dc90 2024-04-25T16:27:26.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.381+0000 7fa51ae20640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa5140b52c0 con 0x7fa5140b6cb0 2024-04-25T16:27:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.379+0000 7faf5c7ce640 1 -- start start 2024-04-25T16:27:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.383+0000 7faf5c7ce640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faf4c007370 0x7faf4c007780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.383+0000 7faf5c7ce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c0516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.383+0000 7faf5c7ce640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faf4c051be0 0x7faf4c051ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.383+0000 7faf5c7ce640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faf4c0097b0 con 0x7faf4c051be0 2024-04-25T16:27:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.383+0000 7faf5c7ce640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faf4c008fb0 con 0x7faf4c007370 2024-04-25T16:27:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.383+0000 7faf5c7ce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c009230 con 0x7faf4c051290 2024-04-25T16:27:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.384+0000 7faf5a56a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faf4c007370 0x7faf4c007780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.384+0000 7faf5a56a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faf4c007370 0x7faf4c007780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41332/0 (socket says 172.21.15.94:41332) 2024-04-25T16:27:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.384+0000 7faf5a56a640 1 -- 172.21.15.94:0/2972400110 learned_addr learned my addr 172.21.15.94:0/2972400110 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.385+0000 7faf5ad6b640 1 --2- 172.21.15.94:0/2972400110 >> v2:172.21.15.94:3300/0 conn(0x7faf4c051be0 0x7faf4c051ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.386+0000 7faf5a56a640 1 -- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 msgr2=0x7faf4c0516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.386+0000 7faf59d69640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c0516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.384+0000 7fa5193bd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa51406dc90 0x7fa51415cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:51650/0 (socket says 172.21.15.94:51650) 2024-04-25T16:27:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.385+0000 7fa513fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa514070c10 0x7fa51406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56240/0 (socket says 172.21.15.94:56240) 2024-04-25T16:27:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.385+0000 7fa513fff640 1 -- 172.21.15.94:0/1641403412 learned_addr learned my addr 172.21.15.94:0/1641403412 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.387+0000 7fa518bbc640 1 --2- 172.21.15.94:0/1641403412 >> v2:172.21.15.94:3300/0 conn(0x7fa5140b6cb0 0x7fa5140b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.386+0000 7faf5a56a640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c0516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.386+0000 7faf5a56a640 1 -- 172.21.15.94:0/2972400110 >> v2:172.21.15.94:3300/0 conn(0x7faf4c051be0 msgr2=0x7faf4c051ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.386+0000 7faf5a56a640 1 --2- 172.21.15.94:0/2972400110 >> v2:172.21.15.94:3300/0 conn(0x7faf4c051be0 0x7faf4c051ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.386+0000 7faf5a56a640 1 -- 172.21.15.94:0/2972400110 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c052530 con 0x7faf4c007370 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.388+0000 7faf59d69640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c0516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.388+0000 7faf5ad6b640 1 --2- 172.21.15.94:0/2972400110 >> v2:172.21.15.94:3300/0 conn(0x7faf4c051be0 0x7faf4c051ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.389+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542473343 0 0) 0x7fa514160560 con 0x7fa51406dc90 2024-04-25T16:27:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.390+0000 7faf5a56a640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c007370 0x7faf4c007780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf54077120 tx=0x7faf540a77f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4f06f00b66349b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.389+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa504003420 con 0x7fa51406dc90 2024-04-25T16:27:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.389+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160007603 0 0) 0x7fa5141610e0 con 0x7fa514070c10 2024-04-25T16:27:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.389+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa514160560 con 0x7fa514070c10 2024-04-25T16:27:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.390+0000 7fa518bbc640 1 -- 172.21.15.94:0/1641403412 >> v1:172.21.15.138:6789/0 conn(0x7fa51406dc90 legacy=0x7fa51415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.391+0000 7fa518bbc640 1 -- 172.21.15.94:0/1641403412 >> v1:172.21.15.94:6789/0 conn(0x7fa514070c10 legacy=0x7fa51406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.391+0000 7fa518bbc640 1 -- 172.21.15.94:0/1641403412 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa514160840 con 0x7fa5140b6cb0 2024-04-25T16:27:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.394+0000 7faf59568640 1 -- 172.21.15.94:0/2972400110 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf540b5070 con 0x7faf4c007370 2024-04-25T16:27:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.394+0000 7faf59568640 1 -- 172.21.15.94:0/2972400110 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf540b0680 con 0x7faf4c007370 2024-04-25T16:27:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.393+0000 7fa518bbc640 1 --2- 172.21.15.94:0/1641403412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa5140b4d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5080097a0 tx=0x7fa50802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d6f9a249e3bb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.394+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508037070 con 0x7fa5140b6cb0 2024-04-25T16:27:26.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.394+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa508032440 con 0x7fa5140b6cb0 2024-04-25T16:27:26.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.394+0000 7fa5137fe640 1 -- 172.21.15.94:0/1641403412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50802ae60 con 0x7fa5140b6cb0 2024-04-25T16:27:26.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.396+0000 7faf5c7ce640 1 -- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c007370 msgr2=0x7faf4c007780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.396+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c007370 0x7faf4c007780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf54077120 tx=0x7faf540a77f0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7faf5c7ce640 1 -- 172.21.15.94:0/2972400110 shutdown_connections 2024-04-25T16:27:26.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/2972400110 >> v2:172.21.15.94:3300/0 conn(0x7faf4c051be0 0x7faf4c051ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c0516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/2972400110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c007370 0x7faf4c007780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7faf5c7ce640 1 -- 172.21.15.94:0/2972400110 >> 172.21.15.94:0/2972400110 conn(0x7faf4c055e10 msgr2=0x7faf4c056220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7fa51ae20640 1 -- 172.21.15.94:0/1641403412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 msgr2=0x7fa5140b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.397+0000 7fa51ae20640 1 --2- 172.21.15.94:0/1641403412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa5140b4d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5080097a0 tx=0x7fa50802a110 comp rx=0 tx=0).stop 2024-04-25T16:27:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.398+0000 7faf5c7ce640 1 -- 172.21.15.94:0/2972400110 shutdown_connections 2024-04-25T16:27:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.398+0000 7faf5c7ce640 1 -- 172.21.15.94:0/2972400110 wait complete. 2024-04-25T16:27:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.403+0000 7faf5c7ce640 1 Processor -- start 2024-04-25T16:27:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.404+0000 7faf5c7ce640 1 -- start start 2024-04-25T16:27:26.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.405+0000 7fa51ae20640 1 -- 172.21.15.94:0/1641403412 shutdown_connections 2024-04-25T16:27:26.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.405+0000 7fa51ae20640 1 --2- 172.21.15.94:0/1641403412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa5140b4d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.405+0000 7fa51ae20640 1 -- 172.21.15.94:0/1641403412 >> 172.21.15.94:0/1641403412 conn(0x7fa514073500 msgr2=0x7fa514073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.405+0000 7fa51ae20640 1 -- 172.21.15.94:0/1641403412 shutdown_connections 2024-04-25T16:27:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.405+0000 7fa51ae20640 1 -- 172.21.15.94:0/1641403412 wait complete. 2024-04-25T16:27:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.409+0000 7fa51ae20640 1 Processor -- start 2024-04-25T16:27:26.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.409+0000 7fa51ae20640 1 -- start start 2024-04-25T16:27:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.412+0000 7fa51ae20640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa514070c10 0x7fa51415e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.414+0000 7faf5c7ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 0x7faf4c02c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.415+0000 7fa51ae20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa514142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.415+0000 7faf5a56a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 0x7faf4c02c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.415+0000 7faf5a56a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 0x7faf4c02c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56066/0 (socket says 172.21.15.94:56066) 2024-04-25T16:27:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.415+0000 7faf5a56a640 1 -- 172.21.15.94:0/3167586432 learned_addr learned my addr 172.21.15.94:0/3167586432 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.415+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c033ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.415+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c051be0 0x7faf4c0d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7fa513fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa514142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c050d70 con 0x7faf4c007370 2024-04-25T16:27:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7fa513fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa514142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56076/0 (socket says 172.21.15.94:56076) 2024-04-25T16:27:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7fa513fff640 1 -- 172.21.15.94:0/200886756 learned_addr learned my addr 172.21.15.94:0/200886756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51415e5c0 0x7fa514142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7faf5ad6b640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c051be0 0x7faf4c0d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa514160e00 con 0x7fa5140b6cb0 2024-04-25T16:27:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.416+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c050a90 con 0x7faf4c051be0 2024-04-25T16:27:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.417+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140b70c0 con 0x7fa51415e5c0 2024-04-25T16:27:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.417+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c0494a0 con 0x7faf4c051290 2024-04-25T16:27:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.417+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa514160180 con 0x7fa514070c10 2024-04-25T16:27:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7fa518bbc640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa514070c10 0x7fa51415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7faf5ad6b640 1 -- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 msgr2=0x7faf4c033ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:27:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7faf5ad6b640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c033ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7faf5ad6b640 1 -- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 msgr2=0x7faf4c02c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7faf5ad6b640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 0x7faf4c02c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7fa5193bd640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51415e5c0 0x7fa514142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7fa513fff640 1 -- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa514070c10 msgr2=0x7fa51415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7fa513fff640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa514070c10 0x7fa51415e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7fa513fff640 1 -- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51415e5c0 msgr2=0x7fa514142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.419+0000 7fa513fff640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51415e5c0 0x7fa514142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.418+0000 7faf5ad6b640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c02d2e0 con 0x7faf4c051be0 2024-04-25T16:27:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.419+0000 7fa513fff640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa514145190 con 0x7fa5140b6cb0 2024-04-25T16:27:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.419+0000 7faf5ad6b640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c051be0 0x7faf4c0d3e00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf5000e7f0 tx=0x7faf5000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.419+0000 7faf5a56a640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 0x7faf4c02c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.420+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50017070 con 0x7faf4c051be0 2024-04-25T16:27:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.420+0000 7fa513fff640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa514142660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c5a0 tx=0x7fa50000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.420+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5000c8c0 con 0x7faf4c051be0 2024-04-25T16:27:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.421+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500015020 con 0x7fa5140b6cb0 2024-04-25T16:27:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.421+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c0d19b0 con 0x7faf4c051be0 2024-04-25T16:27:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.421+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c0d1f70 con 0x7faf4c051be0 2024-04-25T16:27:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.421+0000 7fa5193bd640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51415e5c0 0x7fa514142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.421+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa500011870 con 0x7fa5140b6cb0 2024-04-25T16:27:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.422+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5000cc70 con 0x7faf4c051be0 2024-04-25T16:27:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.422+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5141433e0 con 0x7fa5140b6cb0 2024-04-25T16:27:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.425+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141439a0 con 0x7fa5140b6cb0 2024-04-25T16:27:26.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.425+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500011c40 con 0x7fa5140b6cb0 2024-04-25T16:27:26.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.426+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa500018020 con 0x7fa5140b6cb0 2024-04-25T16:27:26.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.426+0000 7fa5117fa640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa4f80745b0 0x7fa4f8076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.427+0000 7fa518bbc640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa4f80745b0 0x7fa4f8076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.428+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa500097030 con 0x7fa5140b6cb0 2024-04-25T16:27:26.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.428+0000 7fa518bbc640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa4f80745b0 0x7fa4f8076ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa508034000 tx=0x7fa50802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.429+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf5001a070 con 0x7faf4c051be0 2024-04-25T16:27:26.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.429+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c005290 con 0x7faf4c051be0 2024-04-25T16:27:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.429+0000 7faf4affd640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faf38074580 0x7faf38076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.430+0000 7faf5a56a640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faf38074580 0x7faf38076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.431+0000 7faf5a56a640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faf38074580 0x7faf38076a90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faf5402e0e0 tx=0x7faf54070030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.431+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faf500955e0 con 0x7faf4c051be0 2024-04-25T16:27:26.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.432+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4005290 con 0x7fa5140b6cb0 2024-04-25T16:27:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.442+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf5005f3c0 con 0x7faf4c051be0 2024-04-25T16:27:26.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.451+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa50005ef80 con 0x7fa5140b6cb0 2024-04-25T16:27:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.468+0000 7f6042622640 1 Processor -- start 2024-04-25T16:27:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.468+0000 7fe362a30640 1 Processor -- start 2024-04-25T16:27:26.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.468+0000 7fe362a30640 1 -- start start 2024-04-25T16:27:26.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.468+0000 7f6042622640 1 -- start start 2024-04-25T16:27:26.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.470+0000 7fe362a30640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3540544f0 0x7fe354008780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.470+0000 7fbf06c34640 1 Processor -- start 2024-04-25T16:27:26.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.470+0000 7fbf06c34640 1 -- start start 2024-04-25T16:27:26.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe362a30640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe354006cb0 0x7fe3540070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7f6042622640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6034007460 0x7f60340ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7f6042622640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60340e3ca0 con 0x7f60340547b0 2024-04-25T16:27:26.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7f6042622640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6034007990 con 0x7f60340e3890 2024-04-25T16:27:26.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7f6042622640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6034007c70 con 0x7f6034007460 2024-04-25T16:27:26.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3540544f0 0x7fe354008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7f6040e1f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f60340547b0 0x7f6034006d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56254/0 (socket says 172.21.15.94:56254) 2024-04-25T16:27:26.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7f6040e1f640 1 -- 172.21.15.94:0/2913037886 learned_addr learned my addr 172.21.15.94:0/2913037886 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3540544f0 0x7fe354008780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56080/0 (socket says 172.21.15.94:56080) 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 -- 172.21.15.94:0/898888294 learned_addr learned my addr 172.21.15.94:0/898888294 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3540090d0 con 0x7fe354050dd0 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe354039a00 con 0x7fe3540544f0 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe3540093b0 con 0x7fe354006cb0 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe36222f640 1 --2- 172.21.15.94:0/898888294 >> v2:172.21.15.138:3300/0 conn(0x7fe354006cb0 0x7fe3540070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 -- 172.21.15.94:0/898888294 >> v2:172.21.15.138:3300/0 conn(0x7fe354006cb0 msgr2=0x7fe3540070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 --2- 172.21.15.94:0/898888294 >> v2:172.21.15.138:3300/0 conn(0x7fe354006cb0 0x7fe3540070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 -- 172.21.15.94:0/898888294 >> v1:172.21.15.138:6789/0 conn(0x7fe354050dd0 legacy=0x7fe3540511e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:27:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.471+0000 7fe361a2e640 1 -- 172.21.15.94:0/898888294 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe354039690 con 0x7fe3540544f0 2024-04-25T16:27:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.472+0000 7f6041e21640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6034007460 0x7f60340ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7fe361a2e640 1 --2- 172.21.15.94:0/898888294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe354008780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe35400f140 tx=0x7fe35802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1097048e1344e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7fbf06c34640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbf00070ef0 0x7fbf000ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7fe360a2c640 1 -- 172.21.15.94:0/898888294 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358037070 con 0x7fe3540544f0 2024-04-25T16:27:26.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7fe360a2c640 1 -- 172.21.15.94:0/898888294 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe358032440 con 0x7fe3540544f0 2024-04-25T16:27:26.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7fe360a2c640 1 -- 172.21.15.94:0/898888294 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358032810 con 0x7fe3540544f0 2024-04-25T16:27:26.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980871905 0 0) 0x7f60340e3ca0 con 0x7f60340547b0 2024-04-25T16:27:26.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.473+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6028003520 con 0x7f60340547b0 2024-04-25T16:27:26.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2773763050 0 0) 0x7f6028003520 con 0x7f60340547b0 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6034007460 msgr2=0x7f60340ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6034007460 0x7f60340ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 >> v1:172.21.15.138:6789/0 conn(0x7f60340e3890 legacy=0x7f60340540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603403c950 con 0x7f60340547b0 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f603c076bb0 con 0x7f60340547b0 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 >> v1:172.21.15.94:6789/0 conn(0x7f60340547b0 legacy=0x7f6034006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7f6033fff640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60280039e0 0x7f6028003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 msgr2=0x7fe354008780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fe362a30640 1 --2- 172.21.15.94:0/898888294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe354008780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe35400f140 tx=0x7fe35802a230 comp rx=0 tx=0).stop 2024-04-25T16:27:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fbf06c34640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbf00161ea0 0x7fbf0015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fbf06c34640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf0015e8b0 con 0x7fbf0015f590 2024-04-25T16:27:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fbf06c34640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbf000ce710 con 0x7fbf00070ef0 2024-04-25T16:27:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fbf06c34640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbf00162680 con 0x7fbf00161ea0 2024-04-25T16:27:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fbf051d1640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbf0015f590 0x7fbf000aecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56266/0 (socket says 172.21.15.94:56266) 2024-04-25T16:27:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.474+0000 7fbf051d1640 1 -- 172.21.15.94:0/3564960781 learned_addr learned my addr 172.21.15.94:0/3564960781 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 shutdown_connections 2024-04-25T16:27:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fe362a30640 1 --2- 172.21.15.94:0/898888294 >> v2:172.21.15.138:3300/0 conn(0x7fe354006cb0 0x7fe3540070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fe362a30640 1 --2- 172.21.15.94:0/898888294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe354008780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 >> 172.21.15.94:0/898888294 conn(0x7fe354055e00 msgr2=0x7fe354056210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1266103951 0 0) 0x7fbee8001110 con 0x7fbf0015f590 2024-04-25T16:27:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeec003520 con 0x7fbf0015f590 2024-04-25T16:27:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbf049d0640 1 --2- 172.21.15.94:0/3564960781 >> v2:172.21.15.94:3300/0 conn(0x7fbf00070ef0 0x7fbf000ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 shutdown_connections 2024-04-25T16:27:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fe362a30640 1 -- 172.21.15.94:0/898888294 wait complete. 2024-04-25T16:27:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbefffff640 1 --2- 172.21.15.94:0/3564960781 >> v2:172.21.15.138:3300/0 conn(0x7fbf00161ea0 0x7fbf0015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735666681 0 0) 0x7fbeec003520 con 0x7fbf0015f590 2024-04-25T16:27:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 >> v2:172.21.15.138:3300/0 conn(0x7fbf00161ea0 msgr2=0x7fbf0015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbeff7fe640 1 --2- 172.21.15.94:0/3564960781 >> v2:172.21.15.138:3300/0 conn(0x7fbf00161ea0 0x7fbf0015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 >> v2:172.21.15.94:3300/0 conn(0x7fbf00070ef0 msgr2=0x7fbf000ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7f6033fff640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6028004390 0x7f60280048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7f6033fff640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6028005070 0x7f602800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f5820546640 1 Processor -- start 2024-04-25T16:27:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6034007c70 con 0x7f60280039e0 2024-04-25T16:27:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f602800bfe0 con 0x7f6028005070 2024-04-25T16:27:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f602800c1e0 con 0x7f6028004390 2024-04-25T16:27:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.475+0000 7f6041620640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60280039e0 0x7f6028003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f6040e1f640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6028004390 0x7f60280048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f6041620640 1 -- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6028004390 msgr2=0x7f60280048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f5820546640 1 -- start start 2024-04-25T16:27:26.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f6041620640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6028004390 0x7f60280048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f6041620640 1 -- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6028005070 msgr2=0x7f602800b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:27:26.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f6041620640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6028005070 0x7f602800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f6041620640 1 -- 172.21.15.94:0/2913037886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6028003520 con 0x7f60280039e0 2024-04-25T16:27:26.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f6041620640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60280039e0 0x7f6028003e10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6038001850 tx=0x7f6028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d9084eb3fe9704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f5820546640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f581806d580 0x7f581806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038037070 con 0x7f60280039e0 2024-04-25T16:27:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fe362a30640 1 Processor -- start 2024-04-25T16:27:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f896f73f640 1 Processor -- start 2024-04-25T16:27:26.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603802ac10 con 0x7f60280039e0 2024-04-25T16:27:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fe362a30640 1 -- start start 2024-04-25T16:27:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f6033fff640 1 -- 172.21.15.94:0/2913037886 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038001450 con 0x7f60280039e0 2024-04-25T16:27:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f896f73f640 1 -- start start 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f081289f640 1 Processor -- start 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.476+0000 7f081289f640 1 -- start start 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f081289f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f080c06d810 0x7f080c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f081289f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c0af310 0x7f080c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f081289f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0b3c00 con 0x7f080c0af720 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f081289f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f080c06e120 con 0x7f080c06d810 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.478+0000 7f081289f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f080c15d000 con 0x7f080c0af310 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f081209e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f080c0af720 0x7f080c1606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56280/0 (socket says 172.21.15.94:56280) 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f081209e640 1 -- 172.21.15.94:0/187966861 learned_addr learned my addr 172.21.15.94:0/187966861 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728207535 0 0) 0x7f080c0b3c00 con 0x7f080c0af720 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f8003520 con 0x7f080c0af720 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1511434016 0 0) 0x7f07f8003520 con 0x7f080c0af720 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c0af310 msgr2=0x7f080c0b7850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:27:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c0af310 0x7f080c0b7850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 >> v2:172.21.15.138:3300/0 conn(0x7f080c06d810 msgr2=0x7f080c15cac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 --2- 172.21.15.94:0/187966861 >> v2:172.21.15.138:3300/0 conn(0x7f080c06d810 0x7f080c15cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fe362a30640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe354006cb0 0x7fe3540cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 --2- 172.21.15.94:0/3564960781 >> v2:172.21.15.94:3300/0 conn(0x7fbf00070ef0 0x7fbf000ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf00162960 con 0x7fbf0015f590 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fbee8001840 con 0x7fbf0015f590 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 >> v1:172.21.15.94:6789/0 conn(0x7fbf0015f590 legacy=0x7fbf000aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeec003a30 0x7fbeec003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbeec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeec005150 0x7fbeec0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbefffff640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbeec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000ce710 con 0x7fbeec005150 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00162680 con 0x7fbeec004800 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec014980 con 0x7fbeec003a30 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbf049d0640 1 --2- 172.21.15.94:0/3564960781 >> v2:172.21.15.94:3300/0 conn(0x7fbf00070ef0 0x7fbf000ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7fbefffff640 1 -- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeec003a30 msgr2=0x7fbeec003ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7fbefffff640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeec003a30 0x7fbeec003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7fbefffff640 1 -- 172.21.15.94:0/3564960781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeec005150 msgr2=0x7fbeec0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7fbefffff640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeec005150 0x7fbeec0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7fbefffff640 1 -- 172.21.15.94:0/3564960781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec003520 con 0x7fbeec004800 2024-04-25T16:27:26.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fbf051d1640 1 -- 172.21.15.94:0/3564960781 reap_dead start 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f6042622640 1 -- 172.21.15.94:0/2913037886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60280039e0 msgr2=0x7f6028003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f6042622640 1 --2- 172.21.15.94:0/2913037886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60280039e0 0x7f6028003e10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6038001850 tx=0x7f6028005830 comp rx=0 tx=0).stop 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f6041e21640 1 -- 172.21.15.94:0/2913037886 reap_dead start 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f6042622640 1 -- 172.21.15.94:0/2913037886 shutdown_connections 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f6042622640 1 -- 172.21.15.94:0/2913037886 >> 172.21.15.94:0/2913037886 conn(0x7f603403ac50 msgr2=0x7f603403b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f581eae3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f581806d580 0x7f581806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.479+0000 7f581dae1640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f581806f3a0 0x7f58181649c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56288/0 (socket says 172.21.15.94:56288) 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f581e2e2640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f58180afc70 0x7f5818070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:51678/0 (socket says 172.21.15.94:51678) 2024-04-25T16:27:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f581eae3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f581806d580 0x7f581806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56114/0 (socket says 172.21.15.94:56114) 2024-04-25T16:27:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f5820546640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f581806f7b0 con 0x7f581806f3a0 2024-04-25T16:27:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f5820546640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f581806ded0 con 0x7f58180afc70 2024-04-25T16:27:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f5820546640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f58180b4d80 con 0x7f581806d580 2024-04-25T16:27:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fe362a30640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe354050dd0 0x7fe35404b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fbefffff640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbeec004c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbef00073b0 tx=0x7fbeec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbae96f117d53ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fe361a2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe354006cb0 0x7fe3540cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fe361a2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe354006cb0 0x7fe3540cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41416/0 (socket says 172.21.15.94:41416) 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fe361a2e640 1 -- 172.21.15.94:0/487880674 learned_addr learned my addr 172.21.15.94:0/487880674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f896f73f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8968164af0 0x7f896806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f896f73f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f896806dac0 0x7f89680b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f896f73f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896806ded0 con 0x7f896806f3a0 2024-04-25T16:27:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f896f73f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f89680703e0 con 0x7f8968164af0 2024-04-25T16:27:26.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f896f73f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8968070fa0 con 0x7f896806dac0 2024-04-25T16:27:26.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f6042622640 1 -- 172.21.15.94:0/2913037886 shutdown_connections 2024-04-25T16:27:26.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f6042622640 1 -- 172.21.15.94:0/2913037886 wait complete. 2024-04-25T16:27:26.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7fe36122d640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe354050dd0 0x7fe35404b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe35404b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f896d4db640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f896806f3a0 0x7f8968070890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:51690/0 (socket says 172.21.15.94:51690) 2024-04-25T16:27:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f896d4db640 1 -- 172.21.15.94:0/3128864788 learned_addr learned my addr 172.21.15.94:0/3128864788 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe354035530 con 0x7fe3540544f0 2024-04-25T16:27:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f896dcdc640 1 --2- 172.21.15.94:0/3128864788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f896806dac0 0x7f89680b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7fe36222f640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe35404b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f896ccda640 1 --2- 172.21.15.94:0/3128864788 >> v2:172.21.15.94:3300/0 conn(0x7f8968164af0 0x7f896806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f895ffff640 1 -- 172.21.15.94:0/3128864788 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213935511 0 0) 0x7f8964001110 con 0x7f896806f3a0 2024-04-25T16:27:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f895ffff640 1 -- 172.21.15.94:0/3128864788 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8954003520 con 0x7f896806f3a0 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c0b1800 con 0x7f080c0af720 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f0808001650 con 0x7f080c0af720 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 >> v1:172.21.15.94:6789/0 conn(0x7f080c0af720 legacy=0x7f080c1606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f8003a40 0x7f07f8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8004800 0x7f07f8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8005350 0x7f07f80079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f080c15d000 con 0x7f07f8004800 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07f80082e0 con 0x7f07f8003a40 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07f80084e0 con 0x7f07f8005350 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.480+0000 7f081109c640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8004800 0x7f07f8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f081209e640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8005350 0x7f07f80079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f081109c640 1 -- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8005350 msgr2=0x7f07f80079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f081109c640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8005350 0x7f07f80079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f081109c640 1 -- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f8003a40 msgr2=0x7f07f8003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f081109c640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f8003a40 0x7f07f8003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f081109c640 1 -- 172.21.15.94:0/187966861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c0b1800 con 0x7f07f8004800 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7f081109c640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8004800 0x7f07f8004c10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f080c0b1420 tx=0x7f07f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8630c9d54164abc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3540353b0 con 0x7fe354006cb0 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe354008cc0 con 0x7fe354050dd0 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc037070 con 0x7f07f8004800 2024-04-25T16:27:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07fc02a970 con 0x7f07f8004800 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0009c70 con 0x7fbeec004800 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.482+0000 7fbeff7fe640 1 -- 172.21.15.94:0/3564960781 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef003d680 con 0x7fbeec004800 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 -- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 msgr2=0x7fbeec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbeec004c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbef00073b0 tx=0x7fbeec005830 comp rx=0 tx=0).stop 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 -- 172.21.15.94:0/3564960781 shutdown_connections 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 --2- 172.21.15.94:0/3564960781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbeec004c10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 -- 172.21.15.94:0/3564960781 >> 172.21.15.94:0/3564960781 conn(0x7fbf000733f0 msgr2=0x7fbf00073800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 -- 172.21.15.94:0/3564960781 shutdown_connections 2024-04-25T16:27:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fe36222f640 1 -- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe354050dd0 msgr2=0x7fe35404b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fe36222f640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe354050dd0 0x7fe35404b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fe36222f640 1 -- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe354006cb0 msgr2=0x7fe3540cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fe36222f640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe354006cb0 0x7fe3540cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7f896ccda640 1 -- 172.21.15.94:0/3128864788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f896806dac0 msgr2=0x7f89680b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7f896ccda640 1 --2- 172.21.15.94:0/3128864788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f896806dac0 0x7f89680b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7f896ccda640 1 -- 172.21.15.94:0/3128864788 >> v1:172.21.15.138:6789/0 conn(0x7f896806f3a0 legacy=0x7f8968070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7f896ccda640 1 -- 172.21.15.94:0/3128864788 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896806f860 con 0x7f8968164af0 2024-04-25T16:27:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fe36222f640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3540d0330 con 0x7fe3540544f0 2024-04-25T16:27:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f896ccda640 1 --2- 172.21.15.94:0/3128864788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8968164af0 0x7f896806d580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8960009d90 tx=0x7f896002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86119871c21a7a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.481+0000 7f581dae1640 1 -- 172.21.15.94:0/1862721429 learned_addr learned my addr 172.21.15.94:0/1862721429 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f6042622640 1 Processor -- start 2024-04-25T16:27:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f895ffff640 1 -- 172.21.15.94:0/3128864788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960007740 con 0x7f8968164af0 2024-04-25T16:27:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f895ffff640 1 -- 172.21.15.94:0/3128864788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89600078e0 con 0x7f8968164af0 2024-04-25T16:27:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f895ffff640 1 -- 172.21.15.94:0/3128864788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960007740 con 0x7f8968164af0 2024-04-25T16:27:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7fe36222f640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe35404b720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe35c06b9a0 tx=0x7fe35c069c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7fe36122d640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe354050dd0 0x7fe35404b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c06d900 con 0x7fe3540544f0 2024-04-25T16:27:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35c06daa0 con 0x7fe3540544f0 2024-04-25T16:27:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe354049f50 con 0x7fe3540544f0 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f081289f640 1 -- 172.21.15.94:0/187966861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8004800 msgr2=0x7f07f8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f081089b640 1 -- 172.21.15.94:0/187966861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc037070 con 0x7f07f8004800 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f081289f640 1 --2- 172.21.15.94:0/187966861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8004800 0x7f07f8004c10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f080c0b1420 tx=0x7f07f8005830 comp rx=0 tx=0).stop 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f081209e640 1 -- 172.21.15.94:0/187966861 reap_dead start 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f081289f640 1 -- 172.21.15.94:0/187966861 shutdown_connections 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f081289f640 1 -- 172.21.15.94:0/187966861 >> 172.21.15.94:0/187966861 conn(0x7f080c073bd0 msgr2=0x7f080c09b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7f581eae3640 1 -- 172.21.15.94:0/1862721429 >> v1:172.21.15.138:6789/0 conn(0x7f58180afc70 legacy=0x7f5818070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7f581eae3640 1 -- 172.21.15.94:0/1862721429 >> v1:172.21.15.94:6789/0 conn(0x7f581806f3a0 legacy=0x7f58181649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.485+0000 7f581eae3640 1 -- 172.21.15.94:0/1862721429 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581806fa30 con 0x7f581806d580 2024-04-25T16:27:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f581eae3640 1 --2- 172.21.15.94:0/1862721429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806d580 0x7f581806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f581000b050 tx=0x7f581002a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d906b088a84891b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.484+0000 7f6042622640 1 -- start start 2024-04-25T16:27:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.483+0000 7fbf06c34640 1 -- 172.21.15.94:0/3564960781 wait complete. 2024-04-25T16:27:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7fbf06c34640 1 Processor -- start 2024-04-25T16:27:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f081289f640 1 -- 172.21.15.94:0/187966861 shutdown_connections 2024-04-25T16:27:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7fbf06c34640 1 -- start start 2024-04-25T16:27:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6042622640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60340de0d0 0x7f60340d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6042622640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 0x7f60340386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6042622640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f603417d730 0x7f603417db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6042622640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f603403e420 con 0x7f60340382e0 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6042622640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f603403cf10 con 0x7f60340de0d0 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6042622640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f603403e5a0 con 0x7f603417d730 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 0x7f60340386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 0x7f60340386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56166/0 (socket says 172.21.15.94:56166) 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 -- 172.21.15.94:0/848142620 learned_addr learned my addr 172.21.15.94:0/848142620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f896f73f640 1 -- 172.21.15.94:0/3128864788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8968164af0 msgr2=0x7f896806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f896f73f640 1 --2- 172.21.15.94:0/3128864788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8968164af0 0x7f896806d580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8960009d90 tx=0x7f896002a610 comp rx=0 tx=0).stop 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f896f73f640 1 -- 172.21.15.94:0/3128864788 shutdown_connections 2024-04-25T16:27:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6041620640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60340de0d0 0x7f60340d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f896f73f640 1 --2- 172.21.15.94:0/3128864788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f896806dac0 0x7f89680b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f896f73f640 1 --2- 172.21.15.94:0/3128864788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8968164af0 0x7f896806d580 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8960009d90 tx=0x7f896002a610 comp rx=0 tx=0).stop 2024-04-25T16:27:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f896f73f640 1 -- 172.21.15.94:0/3128864788 >> 172.21.15.94:0/3128864788 conn(0x7f8968073500 msgr2=0x7f8968073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.486+0000 7f081289f640 1 -- 172.21.15.94:0/187966861 wait complete. 2024-04-25T16:27:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 -- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f603417d730 msgr2=0x7f603417db40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T16:27:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f603417d730 0x7f603417db40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f581d2e0640 1 -- 172.21.15.94:0/1862721429 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100315f0 con 0x7f581806d580 2024-04-25T16:27:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 -- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60340de0d0 msgr2=0x7f60340d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60340de0d0 0x7f60340d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7f581d2e0640 1 -- 172.21.15.94:0/1862721429 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5810036070 con 0x7f581806d580 2024-04-25T16:27:26.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7f581d2e0640 1 -- 172.21.15.94:0/1862721429 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100319f0 con 0x7f581806d580 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.487+0000 7f6040e1f640 1 -- 172.21.15.94:0/848142620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6034038c30 con 0x7f60340382e0 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7f6040e1f640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 0x7f60340386f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f603c0514f0 tx=0x7f603c0758c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c06dbe0 con 0x7f60340382e0 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603c06dd80 con 0x7f60340382e0 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c096620 con 0x7f60340382e0 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7fbf06c34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbf0009e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.488+0000 7fbf06c34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf002144d0 0x7fbf000cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf06c34640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 0x7fbf0015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf06c34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001622b0 con 0x7fbf002144d0 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf06c34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0015eb30 con 0x7fbeec004800 2024-04-25T16:27:26.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf06c34640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf00162430 con 0x7fbf001fd660 2024-04-25T16:27:26.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60340dbfa0 con 0x7f60340382e0 2024-04-25T16:27:26.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35404a510 con 0x7fe3540544f0 2024-04-25T16:27:26.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f5820546640 1 -- 172.21.15.94:0/1862721429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806d580 msgr2=0x7f581806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f5820546640 1 --2- 172.21.15.94:0/1862721429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806d580 0x7f581806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f581000b050 tx=0x7f581002a8a0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f5820546640 1 -- 172.21.15.94:0/1862721429 shutdown_connections 2024-04-25T16:27:26.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f5820546640 1 --2- 172.21.15.94:0/1862721429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806d580 0x7f581806d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f5820546640 1 -- 172.21.15.94:0/1862721429 >> 172.21.15.94:0/1862721429 conn(0x7f5818073500 msgr2=0x7f5818073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf051d1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 0x7fbf0015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c06de50 con 0x7fe3540544f0 2024-04-25T16:27:26.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe35c094030 con 0x7fe3540544f0 2024-04-25T16:27:26.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf051d1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 0x7fbf0015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:59536/0 (socket says 172.21.15.94:59536) 2024-04-25T16:27:26.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7fbf051d1640 1 -- 172.21.15.94:0/587006765 learned_addr learned my addr 172.21.15.94:0/587006765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7f896f73f640 1 -- 172.21.15.94:0/3128864788 shutdown_connections 2024-04-25T16:27:26.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fe3527fc640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe340074720 0x7fe340076c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fe361a2e640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe340074720 0x7fe340076c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f081289f640 1 Processor -- start 2024-04-25T16:27:26.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.489+0000 7f081289f640 1 -- start start 2024-04-25T16:27:26.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe35c0d6df0 con 0x7fe3540544f0 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fbefffff640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf002144d0 0x7fbf000cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fbf051d1640 1 -- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 msgr2=0x7fbf0009e440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fbf051d1640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbf0009e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fbf051d1640 1 -- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf002144d0 msgr2=0x7fbf000cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fbf051d1640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf002144d0 0x7fbf000cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7fbf051d1640 1 -- 172.21.15.94:0/587006765 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf00214cb0 con 0x7fbf001fd660 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fbf051d1640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 0x7fbf0015b600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf00072d80 tx=0x7fbee800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004bf0 con 0x7fbf001fd660 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf00214f90 con 0x7fbf001fd660 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf000d0220 con 0x7fbf001fd660 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8004d90 con 0x7fbf001fd660 2024-04-25T16:27:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8007270 con 0x7fbf001fd660 2024-04-25T16:27:26.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60340dd5f0 con 0x7f60340382e0 2024-04-25T16:27:26.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7fe361a2e640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe340074720 0x7fe340076c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe358009630 tx=0x7fe35802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7f081289f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f080c15b330 0x7f080c09fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.491+0000 7f081289f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c1f2180 0x7f080c0d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081289f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c1f2590 0x7f080c0d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081289f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f080c06dc20 con 0x7f080c1f2590 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081289f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f080c161490 con 0x7f080c15b330 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081289f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f080c06e5b0 con 0x7f080c1f2180 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081189d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f080c15b330 0x7f080c09fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081189d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f080c15b330 0x7f080c09fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41434/0 (socket says 172.21.15.94:41434) 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081189d640 1 -- 172.21.15.94:0/121726647 learned_addr learned my addr 172.21.15.94:0/121726647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f081209e640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c1f2590 0x7f080c0d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe344005290 con 0x7fe3540544f0 2024-04-25T16:27:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f5820546640 1 -- 172.21.15.94:0/1862721429 shutdown_connections 2024-04-25T16:27:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f5820546640 1 -- 172.21.15.94:0/1862721429 wait complete. 2024-04-25T16:27:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.490+0000 7f896f73f640 1 -- 172.21.15.94:0/3128864788 wait complete. 2024-04-25T16:27:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f896f73f640 1 Processor -- start 2024-04-25T16:27:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f896f73f640 1 -- start start 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f081209e640 1 -- 172.21.15.94:0/121726647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c1f2180 msgr2=0x7f080c0d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f081209e640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c1f2180 0x7f080c0d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f081209e640 1 -- 172.21.15.94:0/121726647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f080c15b330 msgr2=0x7f080c09fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f081209e640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f080c15b330 0x7f080c09fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f081209e640 1 -- 172.21.15.94:0/121726647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c09c420 con 0x7f080c1f2590 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f081209e640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c1f2590 0x7f080c0d29e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ced60 tx=0x7f080801fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.494+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080048a0 con 0x7f080c1f2590 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.494+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0808029070 con 0x7f080c1f2590 2024-04-25T16:27:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.494+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808004c60 con 0x7f080c1f2590 2024-04-25T16:27:26.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.494+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf00076bb0 con 0x7fbf001fd660 2024-04-25T16:27:26.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.492+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f603c0967c0 con 0x7f60340382e0 2024-04-25T16:27:26.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f6031ffb640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6020074500 0x7f6020076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f603c0a0de0 con 0x7f60340382e0 2024-04-25T16:27:26.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.493+0000 7f6041620640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6020074500 0x7f6020076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.494+0000 7f6041620640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6020074500 0x7f6020076a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6038034000 tx=0x7f603802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.496+0000 7f5820546640 1 Processor -- start 2024-04-25T16:27:26.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.496+0000 7f5820546640 1 -- start start 2024-04-25T16:27:26.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.495+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c1f2a90 con 0x7f080c1f2590 2024-04-25T16:27:26.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.495+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c1f3050 con 0x7f080c1f2590 2024-04-25T16:27:26.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.496+0000 7f896f73f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 0x7f89680cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.496+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60340059c0 con 0x7f60340382e0 2024-04-25T16:27:26.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896f73f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f896809c6a0 0x7f89680cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896d4db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 0x7f89680cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896d4db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 0x7f89680cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56186/0 (socket says 172.21.15.94:56186) 2024-04-25T16:27:26.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbee8007410 con 0x7fbf001fd660 2024-04-25T16:27:26.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896d4db640 1 -- 172.21.15.94:0/470326942 learned_addr learned my addr 172.21.15.94:0/470326942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c076bb0 con 0x7f080c1f2590 2024-04-25T16:27:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896ccda640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f896809c6a0 0x7f89680cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89682034e0 0x7f89682038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8968070100 con 0x7f896806dac0 2024-04-25T16:27:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f896806fe20 con 0x7f896809c6a0 2024-04-25T16:27:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f89680afc70 con 0x7f89682034e0 2024-04-25T16:27:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7fbefd7fa640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbee00745b0 0x7fbee0076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.497+0000 7f896dcdc640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89682034e0 0x7f89682038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbee8094ce0 con 0x7fbf001fd660 2024-04-25T16:27:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7fbf049d0640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbee00745b0 0x7fbee0076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7f5820546640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 0x7f58181576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7f896d4db640 1 -- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89682034e0 msgr2=0x7f89682038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7f5820546640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58180afc70 0x7f5818162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7f581e2e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 0x7f58181576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7f581e2e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 0x7f58181576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56200/0 (socket says 172.21.15.94:56200) 2024-04-25T16:27:26.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7fbf049d0640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbee00745b0 0x7fbee0076ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf000520f0 tx=0x7fbef8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.498+0000 7f896d4db640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89682034e0 0x7f89682038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f896d4db640 1 -- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f896809c6a0 msgr2=0x7f89680cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f896d4db640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f896809c6a0 0x7f89680cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f581e2e2640 1 -- 172.21.15.94:0/2116357550 learned_addr learned my addr 172.21.15.94:0/2116357550 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f896d4db640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8968204240 con 0x7f896806dac0 2024-04-25T16:27:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818158ee0 0x7f58181592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f581dae1640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58180afc70 0x7f5818162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58180702d0 con 0x7f581806f3a0 2024-04-25T16:27:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f896d4db640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 0x7f89680cf560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f896400c400 tx=0x7f896400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f896ccda640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f896809c6a0 0x7f89680cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f581eae3640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818158ee0 0x7f58181592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f581806fff0 con 0x7f5818158ee0 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.499+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f581816f820 con 0x7f58180afc70 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581e2e2640 1 -- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58180afc70 msgr2=0x7f5818162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581e2e2640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58180afc70 0x7f5818162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581e2e2640 1 -- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818158ee0 msgr2=0x7f58181592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581e2e2640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818158ee0 0x7f58181592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581e2e2640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5818159830 con 0x7f581806f3a0 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581dae1640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58180afc70 0x7f5818162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581eae3640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818158ee0 0x7f58181592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.500+0000 7f581e2e2640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 0x7f58181576d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f581400b440 tx=0x7f581400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.501+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf0015e8b0 con 0x7fbf001fd660 2024-04-25T16:27:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.504+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe35c144b80 con 0x7fe3540544f0 2024-04-25T16:27:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.505+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814015020 con 0x7f581806f3a0 2024-04-25T16:27:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.505+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5818165de0 con 0x7f581806f3a0 2024-04-25T16:27:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.505+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181663a0 con 0x7f581806f3a0 2024-04-25T16:27:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.506+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964017070 con 0x7f896806dac0 2024-04-25T16:27:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.506+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896809bbc0 con 0x7f896806dac0 2024-04-25T16:27:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.506+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896809c180 con 0x7f896806dac0 2024-04-25T16:27:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.507+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08080060d0 con 0x7f080c1f2590 2024-04-25T16:27:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.507+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818075650 con 0x7f581806f3a0 2024-04-25T16:27:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.508+0000 7f08027fc640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07f00744e0 0x7f07f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.508+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896809b9b0 con 0x7f896806dac0 2024-04-25T16:27:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.508+0000 7f081189d640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07f00744e0 0x7f07f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.508+0000 7f081189d640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07f00744e0 0x7f07f00769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f080c1f6880 tx=0x7f080400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.515+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0808004e00 con 0x7f080c1f2590 2024-04-25T16:27:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.515+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964005ce0 con 0x7f896806dac0 2024-04-25T16:27:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.515+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89640072d0 con 0x7f896806dac0 2024-04-25T16:27:26.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.515+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5814011890 con 0x7f581806f3a0 2024-04-25T16:27:26.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.515+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6034038c30 con 0x7f60340382e0 2024-04-25T16:27:26.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.515+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814015020 con 0x7f581806f3a0 2024-04-25T16:27:26.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.525+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0808071070 con 0x7f080c1f2590 2024-04-25T16:27:26.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.528+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f896401a070 con 0x7f896806dac0 2024-04-25T16:27:26.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.528+0000 7f895dffb640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f894c0744e0 0x7f894c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.528+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5814018020 con 0x7f581806f3a0 2024-04-25T16:27:26.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.529+0000 7f580effd640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57fc0748b0 0x7f57fc076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:26.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.529+0000 7f896ccda640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f894c0744e0 0x7f894c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.529+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8964013df0 con 0x7f896806dac0 2024-04-25T16:27:26.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.529+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5814011690 con 0x7f581806f3a0 2024-04-25T16:27:26.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.530+0000 7f581dae1640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57fc0748b0 0x7f57fc076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:26.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.530+0000 7f581dae1640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57fc0748b0 0x7f57fc076dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f581806f350 tx=0x7f5808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.530+0000 7f896ccda640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f894c0744e0 0x7f894c0769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8960034000 tx=0x7f896002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:26.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.537+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8964021020 con 0x7f896806dac0 2024-04-25T16:27:26.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.547+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5818075650 con 0x7f581806f3a0 2024-04-25T16:27:26.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.843+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7faf3c005580 con 0x7faf4c051be0 2024-04-25T16:27:26.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.844+0000 7faf4affd640 1 -- 172.21.15.94:0/3167586432 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf500636c0 con 0x7faf4c051be0 2024-04-25T16:27:26.847 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:27:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.847+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faf38074580 msgr2=0x7faf38076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.847+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faf38074580 0x7faf38076a90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faf5402e0e0 tx=0x7faf54070030 comp rx=0 tx=0).stop 2024-04-25T16:27:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.847+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c051be0 msgr2=0x7faf4c0d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.847+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c051be0 0x7faf4c0d3e00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf5000e7f0 tx=0x7faf5000ecb0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 shutdown_connections 2024-04-25T16:27:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7faf38074580 0x7faf38076a90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c051be0 0x7faf4c0d3e00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf4c051290 0x7faf4c033ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 --2- 172.21.15.94:0/3167586432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf4c007370 0x7faf4c02c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 >> 172.21.15.94:0/3167586432 conn(0x7faf4c055e10 msgr2=0x7faf4c04e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 shutdown_connections 2024-04-25T16:27:26.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.848+0000 7faf5c7ce640 1 -- 172.21.15.94:0/3167586432 wait complete. 2024-04-25T16:27:26.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T16:27:26.865 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.869+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa4f4005850 con 0x7fa5140b6cb0 2024-04-25T16:27:26.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.870+0000 7fa5117fa640 1 -- 172.21.15.94:0/200886756 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa50005e880 con 0x7fa5140b6cb0 2024-04-25T16:27:26.870 INFO:teuthology.orchestra.run.smithi094.stdout:60129542157 2024-04-25T16:27:26.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.873+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa4f80745b0 msgr2=0x7fa4f8076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.873+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa4f80745b0 0x7fa4f8076ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa508034000 tx=0x7fa50802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.873+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 msgr2=0x7fa514142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.873+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa514142660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c5a0 tx=0x7fa50000c9a0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 shutdown_connections 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa4f80745b0 0x7fa4f8076ac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51415e5c0 0x7fa514142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b6cb0 0x7fa514142660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 --2- 172.21.15.94:0/200886756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa514070c10 0x7fa51415e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 >> 172.21.15.94:0/200886756 conn(0x7fa514073500 msgr2=0x7fa514156770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 shutdown_connections 2024-04-25T16:27:26.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.874+0000 7fa51ae20640 1 -- 172.21.15.94:0/200886756 wait complete. 2024-04-25T16:27:26.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542157 for osd.7 2024-04-25T16:27:26.891 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.896+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe344005850 con 0x7fe3540544f0 2024-04-25T16:27:26.897 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:27:26.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.897+0000 7fe3527fc640 1 -- 172.21.15.94:0/487880674 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe35c0db260 con 0x7fe3540544f0 2024-04-25T16:27:26.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.898+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6034056dc0 con 0x7f60340382e0 2024-04-25T16:27:26.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.898+0000 7f6031ffb640 1 -- 172.21.15.94:0/848142620 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f603c0dc800 con 0x7f60340382e0 2024-04-25T16:27:26.899 INFO:teuthology.orchestra.run.smithi094.stdout:60129542156 2024-04-25T16:27:26.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbf00214a40 con 0x7fbf001fd660 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe340074720 msgr2=0x7fe340076c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe340074720 0x7fe340076c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe358009630 tx=0x7fe35802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 msgr2=0x7fe35404b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe35404b720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe35c06b9a0 tx=0x7fe35c069c10 comp rx=0 tx=0).stop 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 shutdown_connections 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe340074720 0x7fe340076c30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3540544f0 0x7fe35404b720 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe354050dd0 0x7fe35404b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 --2- 172.21.15.94:0/487880674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe354006cb0 0x7fe3540cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 >> 172.21.15.94:0/487880674 conn(0x7fe354055e00 msgr2=0x7fe35403b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6020074500 msgr2=0x7f6020076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7f6042622640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6020074500 0x7f6020076a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6038034000 tx=0x7f603802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 msgr2=0x7f60340386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7f6042622640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 0x7f60340386f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f603c0514f0 tx=0x7f603c0758c0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.900+0000 7fbefd7fa640 1 -- 172.21.15.94:0/587006765 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbee805f1a0 con 0x7fbf001fd660 2024-04-25T16:27:26.903 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:27:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 shutdown_connections 2024-04-25T16:27:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6020074500 0x7f6020076a10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f603417d730 0x7f603417db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60340382e0 0x7f60340386f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 --2- 172.21.15.94:0/848142620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60340de0d0 0x7f60340d30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 >> 172.21.15.94:0/848142620 conn(0x7f603403ac50 msgr2=0x7f6034053e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 shutdown_connections 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7fe362a30640 1 -- 172.21.15.94:0/487880674 wait complete. 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 shutdown_connections 2024-04-25T16:27:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.901+0000 7f6042622640 1 -- 172.21.15.94:0/848142620 wait complete. 2024-04-25T16:27:26.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbee00745b0 msgr2=0x7fbee0076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbee00745b0 0x7fbee0076ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf000520f0 tx=0x7fbef8009190 comp rx=0 tx=0).stop 2024-04-25T16:27:26.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 msgr2=0x7fbf0015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 0x7fbf0015b600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf00072d80 tx=0x7fbee800c820 comp rx=0 tx=0).stop 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 shutdown_connections 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbee00745b0 0x7fbee0076ac0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf001fd660 0x7fbf0015b600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf002144d0 0x7fbf000cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 --2- 172.21.15.94:0/587006765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeec004800 0x7fbf0009e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 >> 172.21.15.94:0/587006765 conn(0x7fbf000733f0 msgr2=0x7fbf000b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 shutdown_connections 2024-04-25T16:27:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.903+0000 7fbf06c34640 1 -- 172.21.15.94:0/587006765 wait complete. 2024-04-25T16:27:26.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.6 2024-04-25T16:27:26.917 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-25T16:27:26.917 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T16:27:26.920 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.934+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f896806a180 con 0x7f896806dac0 2024-04-25T16:27:26.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.934+0000 7f895dffb640 1 -- 172.21.15.94:0/470326942 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f896806ded0 con 0x7f896806dac0 2024-04-25T16:27:26.934 INFO:teuthology.orchestra.run.smithi094.stdout:60129542156 2024-04-25T16:27:26.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f894c0744e0 msgr2=0x7f894c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f894c0744e0 0x7f894c0769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8960034000 tx=0x7f896002f000 comp rx=0 tx=0).stop 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 msgr2=0x7f89680cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 0x7f89680cf560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f896400c400 tx=0x7f896400c8c0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 shutdown_connections 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f894c0744e0 0x7f894c0769f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89682034e0 0x7f89682038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f896809c6a0 0x7f89680cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 --2- 172.21.15.94:0/470326942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f896806dac0 0x7f89680cf560 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 >> 172.21.15.94:0/470326942 conn(0x7f8968073500 msgr2=0x7f89680b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 shutdown_connections 2024-04-25T16:27:26.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.936+0000 7f896f73f640 1 -- 172.21.15.94:0/470326942 wait complete. 2024-04-25T16:27:26.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.946+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f080c0d0300 con 0x7f080c1f2590 2024-04-25T16:27:26.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.946+0000 7f08027fc640 1 -- 172.21.15.94:0/121726647 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0808070970 con 0x7f080c1f2590 2024-04-25T16:27:26.947 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:27:26.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.4 2024-04-25T16:27:26.948 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07f00744e0 msgr2=0x7f07f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07f00744e0 0x7f07f00769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f080c1f6880 tx=0x7f080400b040 comp rx=0 tx=0).stop 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c1f2590 msgr2=0x7f080c0d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c1f2590 0x7f080c0d29e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ced60 tx=0x7f080801fc10 comp rx=0 tx=0).stop 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 shutdown_connections 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07f00744e0 0x7f07f00769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c1f2590 0x7f080c0d29e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080c1f2180 0x7f080c0d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 --2- 172.21.15.94:0/121726647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f080c15b330 0x7f080c09fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.948+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 >> 172.21.15.94:0/121726647 conn(0x7f080c073bd0 msgr2=0x7f080c160400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.949+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 shutdown_connections 2024-04-25T16:27:26.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.949+0000 7f081289f640 1 -- 172.21.15.94:0/121726647 wait complete. 2024-04-25T16:27:26.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.957+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f581806a180 con 0x7f581806f3a0 2024-04-25T16:27:26.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.958+0000 7f580effd640 1 -- 172.21.15.94:0/2116357550 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f581401f020 con 0x7f581806f3a0 2024-04-25T16:27:26.958 INFO:teuthology.orchestra.run.smithi094.stdout:60129542156 2024-04-25T16:27:26.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-04-25T16:27:26.961 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57fc0748b0 msgr2=0x7f57fc076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57fc0748b0 0x7f57fc076dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f581806f350 tx=0x7f5808009040 comp rx=0 tx=0).stop 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 msgr2=0x7f58181576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 0x7f58181576d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f581400b440 tx=0x7f581400b900 comp rx=0 tx=0).stop 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 shutdown_connections 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57fc0748b0 0x7f57fc076dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818158ee0 0x7f58181592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58180afc70 0x7f5818162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 --2- 172.21.15.94:0/2116357550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581806f3a0 0x7f58181576d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:26.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 >> 172.21.15.94:0/2116357550 conn(0x7f5818073500 msgr2=0x7f58180b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:26.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 shutdown_connections 2024-04-25T16:27:26.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:26.960+0000 7f5820546640 1 -- 172.21.15.94:0/2116357550 wait complete. 2024-04-25T16:27:26.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.5 2024-04-25T16:27:26.971 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:26.971 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T16:27:26.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:27.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.118+0000 7fd73f0aa640 1 Processor -- start 2024-04-25T16:27:27.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.118+0000 7fd73f0aa640 1 -- start start 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73f0aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd7380b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73f0aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd73806ef10 0x7fd7380d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73f0aa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7380d13f0 con 0x7fd7380d0fe0 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73f0aa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd7380b50e0 con 0x7fd73806ef10 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73f0aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd7380b42a0 con 0x7fd7380b3570 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd737fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd73806ef10 0x7fd7380d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73d647640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd7380d0fe0 0x7fd73808d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56304/0 (socket says 172.21.15.94:56304) 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73d647640 1 -- 172.21.15.94:0/913795651 learned_addr learned my addr 172.21.15.94:0/913795651 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd737fff640 1 -- 172.21.15.94:0/913795651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 msgr2=0x7fd7380b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73ce46640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd7380b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd737fff640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd7380b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd737fff640 1 -- 172.21.15.94:0/913795651 >> v1:172.21.15.94:6789/0 conn(0x7fd7380d0fe0 legacy=0x7fd73808d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd7377fe640 1 -- 172.21.15.94:0/913795651 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803563446 0 0) 0x7fd730001490 con 0x7fd7380d0fe0 2024-04-25T16:27:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd737fff640 1 -- 172.21.15.94:0/913795651 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7380b44d0 con 0x7fd73806ef10 2024-04-25T16:27:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.120+0000 7fd73ce46640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd7380b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.121+0000 7fd737fff640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380d0aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd7240097a0 tx=0x7fd72402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59191e8a1be51587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.121+0000 7fd7377fe640 1 -- 172.21.15.94:0/913795651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724037070 con 0x7fd73806ef10 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.121+0000 7fd7377fe640 1 -- 172.21.15.94:0/913795651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724032440 con 0x7fd73806ef10 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.121+0000 7fd7377fe640 1 -- 172.21.15.94:0/913795651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724032810 con 0x7fd73806ef10 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 -- 172.21.15.94:0/913795651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 msgr2=0x7fd7380d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380d0aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd7240097a0 tx=0x7fd72402a210 comp rx=0 tx=0).stop 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 -- 172.21.15.94:0/913795651 shutdown_connections 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380d0aa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/913795651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd7380b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 -- 172.21.15.94:0/913795651 >> 172.21.15.94:0/913795651 conn(0x7fd7380d4090 msgr2=0x7fd7380d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 -- 172.21.15.94:0/913795651 shutdown_connections 2024-04-25T16:27:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.122+0000 7fd73f0aa640 1 -- 172.21.15.94:0/913795651 wait complete. 2024-04-25T16:27:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.123+0000 7fd73f0aa640 1 Processor -- start 2024-04-25T16:27:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.123+0000 7fd73f0aa640 1 -- start start 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73f0aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73f0aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd73814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73f0aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7380d0fe0 0x7fd73814d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73f0aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7380b4d70 con 0x7fd73806ef10 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73f0aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7380b4a90 con 0x7fd7380d0fe0 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73f0aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd73806f320 con 0x7fd7380b3570 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56222/0 (socket says 172.21.15.94:56222) 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 -- 172.21.15.94:0/3684653788 learned_addr learned my addr 172.21.15.94:0/3684653788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd737fff640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd73814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73d647640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7380d0fe0 0x7fd73814d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 -- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 msgr2=0x7fd73814d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd73814d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 -- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7380d0fe0 msgr2=0x7fd73814d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7380d0fe0 0x7fd73814d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 -- 172.21.15.94:0/3684653788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7380c86d0 con 0x7fd73806ef10 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd737fff640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd73814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73d647640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7380d0fe0 0x7fd73814d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.125+0000 7fd73ce46640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380c9c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c820 tx=0x7fd72c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.126+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004220 con 0x7fd73806ef10 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.126+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c0043c0 con 0x7fd73806ef10 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.126+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c005bb0 con 0x7fd73806ef10 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.126+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7380c8bd0 con 0x7fd73806ef10 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.126+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7380c9190 con 0x7fd73806ef10 2024-04-25T16:27:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.127+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73806a180 con 0x7fd73806ef10 2024-04-25T16:27:27.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.127+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd72c0077d0 con 0x7fd73806ef10 2024-04-25T16:27:27.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.128+0000 7fd7357fa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd71c0744e0 0x7fd71c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.128+0000 7fd737fff640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd71c0744e0 0x7fd71c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.128+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd72c095810 con 0x7fd73806ef10 2024-04-25T16:27:27.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.128+0000 7fd737fff640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd71c0744e0 0x7fd71c0769f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd724034000 tx=0x7fd72402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.133+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd72c01e020 con 0x7fd73806ef10 2024-04-25T16:27:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.304+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7380ca1b0 con 0x7fd71c0744e0 2024-04-25T16:27:27.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.312+0000 7fd7357fa640 1 -- 172.21.15.94:0/3684653788 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332520 (secure 0 0 0) 0x7fd7380ca1b0 con 0x7fd71c0744e0 2024-04-25T16:27:27.316 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:27.318 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd71c0744e0 msgr2=0x7fd71c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd71c0744e0 0x7fd71c0769f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd724034000 tx=0x7fd72402e000 comp rx=0 tx=0).stop 2024-04-25T16:27:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 msgr2=0x7fd7380c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380c9c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c820 tx=0x7fd72c00cce0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 shutdown_connections 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd71c0744e0 0x7fd71c0769f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7380d0fe0 0x7fd73814d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7380b3570 0x7fd73814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 --2- 172.21.15.94:0/3684653788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd73806ef10 0x7fd7380c9c70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.315+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 >> 172.21.15.94:0/3684653788 conn(0x7fd7380d4090 msgr2=0x7fd7380b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.316+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 shutdown_connections 2024-04-25T16:27:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.316+0000 7fd73f0aa640 1 -- 172.21.15.94:0/3684653788 wait complete. 2024-04-25T16:27:27.328 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-25T16:27:27.304620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236072,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738648,"statfs":{"total":773094113280,"available":772852375552,"internally_reserved":0,"allocated":3350528,"data_stored":2020472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003276"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901527+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:02.901527+0000","last_peered":"2024-04-25T16:27:02.901527+0000","last_clean":"2024-04-25T16:27:02.901527+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:02.901527+0000","last_undegraded":"2024-04-25T16:27:02.901527+0000","last_fullsized":"2024-04-25T16:27:02.901527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584607+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:01.584607+0000","last_peered":"2024-04-25T16:27:01.584607+0000","last_clean":"2024-04-25T16:27:01.584607+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:01.584607+0000","last_undegraded":"2024-04-25T16:27:01.584607+0000","last_fullsized":"2024-04-25T16:27:01.584607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584101+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:01.584101+0000","last_peered":"2024-04-25T16:27:01.584101+0000","last_clean":"2024-04-25T16:27:01.584101+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:01.584101+0000","last_undegraded":"2024-04-25T16:27:01.584101+0000","last_fullsized":"2024-04-25T16:27:01.584101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584845+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:01.584845+0000","last_peered":"2024-04-25T16:27:01.584845+0000","last_clean":"2024-04-25T16:27:01.584845+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:01.584845+0000","last_undegraded":"2024-04-25T16:27:01.584845+0000","last_fullsized":"2024-04-25T16:27:01.584845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901225+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:02.901225+0000","last_peered":"2024-04-25T16:27:02.901225+0000","last_clean":"2024-04-25T16:27:02.901225+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:02.901225+0000","last_undegraded":"2024-04-25T16:27:02.901225+0000","last_fullsized":"2024-04-25T16:27:02.901225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589204+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:01.589204+0000","last_peered":"2024-04-25T16:27:01.589204+0000","last_clean":"2024-04-25T16:27:01.589204+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:01.589204+0000","last_undegraded":"2024-04-25T16:27:01.589204+0000","last_fullsized":"2024-04-25T16:27:01.589204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900752+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:02.900752+0000","last_peered":"2024-04-25T16:27:02.900752+0000","last_clean":"2024-04-25T16:27:02.900752+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:02.900752+0000","last_undegraded":"2024-04-25T16:27:02.900752+0000","last_fullsized":"2024-04-25T16:27:02.900752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161964+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:02.161964+0000","last_peered":"2024-04-25T16:27:02.161964+0000","last_clean":"2024-04-25T16:27:02.161964+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:02.161964+0000","last_undegraded":"2024-04-25T16:27:02.161964+0000","last_fullsized":"2024-04-25T16:27:02.161964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589641+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:01.589641+0000","last_peered":"2024-04-25T16:27:01.589641+0000","last_clean":"2024-04-25T16:27:01.589641+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:01.589641+0000","last_undegraded":"2024-04-25T16:27:01.589641+0000","last_fullsized":"2024-04-25T16:27:01.589641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900437+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:02.900437+0000","last_peered":"2024-04-25T16:27:02.900437+0000","last_clean":"2024-04-25T16:27:02.900437+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:02.900437+0000","last_undegraded":"2024-04-25T16:27:02.900437+0000","last_fullsized":"2024-04-25T16:27:02.900437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162051+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:02.162051+0000","last_peered":"2024-04-25T16:27:02.162051+0000","last_clean":"2024-04-25T16:27:02.162051+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:02.162051+0000","last_undegraded":"2024-04-25T16:27:02.162051+0000","last_fullsized":"2024-04-25T16:27:02.162051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589525+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:01.589525+0000","last_peered":"2024-04-25T16:27:01.589525+0000","last_clean":"2024-04-25T16:27:01.589525+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:01.589525+0000","last_undegraded":"2024-04-25T16:27:01.589525+0000","last_fullsized":"2024-04-25T16:27:01.589525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591279+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:01.591279+0000","last_peered":"2024-04-25T16:27:01.591279+0000","last_clean":"2024-04-25T16:27:01.591279+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:01.591279+0000","last_undegraded":"2024-04-25T16:27:01.591279+0000","last_fullsized":"2024-04-25T16:27:01.591279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900534+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:02.900534+0000","last_peered":"2024-04-25T16:27:02.900534+0000","last_clean":"2024-04-25T16:27:02.900534+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:02.900534+0000","last_undegraded":"2024-04-25T16:27:02.900534+0000","last_fullsized":"2024-04-25T16:27:02.900534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591334+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:01.591334+0000","last_peered":"2024-04-25T16:27:01.591334+0000","last_clean":"2024-04-25T16:27:01.591334+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:01.591334+0000","last_undegraded":"2024-04-25T16:27:01.591334+0000","last_fullsized":"2024-04-25T16:27:01.591334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584779+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:01.584779+0000","last_peered":"2024-04-25T16:27:01.584779+0000","last_clean":"2024-04-25T16:27:01.584779+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:01.584779+0000","last_undegraded":"2024-04-25T16:27:01.584779+0000","last_fullsized":"2024-04-25T16:27:01.584779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589901+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:01.589901+0000","last_peered":"2024-04-25T16:27:01.589901+0000","last_clean":"2024-04-25T16:27:01.589901+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:01.589901+0000","last_undegraded":"2024-04-25T16:27:01.589901+0000","last_fullsized":"2024-04-25T16:27:01.589901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584240+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:01.584240+0000","last_peered":"2024-04-25T16:27:01.584240+0000","last_clean":"2024-04-25T16:27:01.584240+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:01.584240+0000","last_undegraded":"2024-04-25T16:27:01.584240+0000","last_fullsized":"2024-04-25T16:27:01.584240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900529+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:02.900529+0000","last_peered":"2024-04-25T16:27:02.900529+0000","last_clean":"2024-04-25T16:27:02.900529+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:02.900529+0000","last_undegraded":"2024-04-25T16:27:02.900529+0000","last_fullsized":"2024-04-25T16:27:02.900529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:27:00.605424+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:27:00.604972+0000","last_became_peered":"2024-04-25T16:27:00.604972+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590785+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:01.590785+0000","last_peered":"2024-04-25T16:27:01.590785+0000","last_clean":"2024-04-25T16:27:01.590785+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:01.590785+0000","last_undegraded":"2024-04-25T16:27:01.590785+0000","last_fullsized":"2024-04-25T16:27:01.590785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900076+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:02.900076+0000","last_peered":"2024-04-25T16:27:02.900076+0000","last_clean":"2024-04-25T16:27:02.900076+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:02.900076+0000","last_undegraded":"2024-04-25T16:27:02.900076+0000","last_fullsized":"2024-04-25T16:27:02.900076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591342+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:01.591342+0000","last_peered":"2024-04-25T16:27:01.591342+0000","last_clean":"2024-04-25T16:27:01.591342+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:01.591342+0000","last_undegraded":"2024-04-25T16:27:01.591342+0000","last_fullsized":"2024-04-25T16:27:01.591342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901614+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:02.901614+0000","last_peered":"2024-04-25T16:27:02.901614+0000","last_clean":"2024-04-25T16:27:02.901614+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:02.901614+0000","last_undegraded":"2024-04-25T16:27:02.901614+0000","last_fullsized":"2024-04-25T16:27:02.901614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591218+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:01.591218+0000","last_peered":"2024-04-25T16:27:01.591218+0000","last_clean":"2024-04-25T16:27:01.591218+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:01.591218+0000","last_undegraded":"2024-04-25T16:27:01.591218+0000","last_fullsized":"2024-04-25T16:27:01.591218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590686+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:01.590686+0000","last_peered":"2024-04-25T16:27:01.590686+0000","last_clean":"2024-04-25T16:27:01.590686+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:01.590686+0000","last_undegraded":"2024-04-25T16:27:01.590686+0000","last_fullsized":"2024-04-25T16:27:01.590686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584723+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:01.584723+0000","last_peered":"2024-04-25T16:27:01.584723+0000","last_clean":"2024-04-25T16:27:01.584723+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:01.584723+0000","last_undegraded":"2024-04-25T16:27:01.584723+0000","last_fullsized":"2024-04-25T16:27:01.584723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901145+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:02.901145+0000","last_peered":"2024-04-25T16:27:02.901145+0000","last_clean":"2024-04-25T16:27:02.901145+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:02.901145+0000","last_undegraded":"2024-04-25T16:27:02.901145+0000","last_fullsized":"2024-04-25T16:27:02.901145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.583975+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:01.583975+0000","last_peered":"2024-04-25T16:27:01.583975+0000","last_clean":"2024-04-25T16:27:01.583975+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:01.583975+0000","last_undegraded":"2024-04-25T16:27:01.583975+0000","last_fullsized":"2024-04-25T16:27:01.583975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900898+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:02.900898+0000","last_peered":"2024-04-25T16:27:02.900898+0000","last_clean":"2024-04-25T16:27:02.900898+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:02.900898+0000","last_undegraded":"2024-04-25T16:27:02.900898+0000","last_fullsized":"2024-04-25T16:27:02.900898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900860+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:02.900860+0000","last_peered":"2024-04-25T16:27:02.900860+0000","last_clean":"2024-04-25T16:27:02.900860+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:02.900860+0000","last_undegraded":"2024-04-25T16:27:02.900860+0000","last_fullsized":"2024-04-25T16:27:02.900860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901651+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:02.901651+0000","last_peered":"2024-04-25T16:27:02.901651+0000","last_clean":"2024-04-25T16:27:02.901651+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:02.901651+0000","last_undegraded":"2024-04-25T16:27:02.901651+0000","last_fullsized":"2024-04-25T16:27:02.901651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162233+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:02.162233+0000","last_peered":"2024-04-25T16:27:02.162233+0000","last_clean":"2024-04-25T16:27:02.162233+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:02.162233+0000","last_undegraded":"2024-04-25T16:27:02.162233+0000","last_fullsized":"2024-04-25T16:27:02.162233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901021+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:02.901021+0000","last_peered":"2024-04-25T16:27:02.901021+0000","last_clean":"2024-04-25T16:27:02.901021+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:02.901021+0000","last_undegraded":"2024-04-25T16:27:02.901021+0000","last_fullsized":"2024-04-25T16:27:02.901021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900663+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:02.900663+0000","last_peered":"2024-04-25T16:27:02.900663+0000","last_clean":"2024-04-25T16:27:02.900663+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:02.900663+0000","last_undegraded":"2024-04-25T16:27:02.900663+0000","last_fullsized":"2024-04-25T16:27:02.900663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590164+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:01.590164+0000","last_peered":"2024-04-25T16:27:01.590164+0000","last_clean":"2024-04-25T16:27:01.590164+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:01.590164+0000","last_undegraded":"2024-04-25T16:27:01.590164+0000","last_fullsized":"2024-04-25T16:27:01.590164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162067+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:02.162067+0000","last_peered":"2024-04-25T16:27:02.162067+0000","last_clean":"2024-04-25T16:27:02.162067+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:02.162067+0000","last_undegraded":"2024-04-25T16:27:02.162067+0000","last_fullsized":"2024-04-25T16:27:02.162067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900915+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:02.900915+0000","last_peered":"2024-04-25T16:27:02.900915+0000","last_clean":"2024-04-25T16:27:02.900915+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:02.900915+0000","last_undegraded":"2024-04-25T16:27:02.900915+0000","last_fullsized":"2024-04-25T16:27:02.900915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900165+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:02.900165+0000","last_peered":"2024-04-25T16:27:02.900165+0000","last_clean":"2024-04-25T16:27:02.900165+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:02.900165+0000","last_undegraded":"2024-04-25T16:27:02.900165+0000","last_fullsized":"2024-04-25T16:27:02.900165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900098+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:02.900098+0000","last_peered":"2024-04-25T16:27:02.900098+0000","last_clean":"2024-04-25T16:27:02.900098+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:02.900098+0000","last_undegraded":"2024-04-25T16:27:02.900098+0000","last_fullsized":"2024-04-25T16:27:02.900098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901408+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:02.901408+0000","last_peered":"2024-04-25T16:27:02.901408+0000","last_clean":"2024-04-25T16:27:02.901408+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:02.901408+0000","last_undegraded":"2024-04-25T16:27:02.901408+0000","last_fullsized":"2024-04-25T16:27:02.901408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901450+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:02.901450+0000","last_peered":"2024-04-25T16:27:02.901450+0000","last_clean":"2024-04-25T16:27:02.901450+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:02.901450+0000","last_undegraded":"2024-04-25T16:27:02.901450+0000","last_fullsized":"2024-04-25T16:27:02.901450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900079+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:02.900079+0000","last_peered":"2024-04-25T16:27:02.900079+0000","last_clean":"2024-04-25T16:27:02.900079+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:02.900079+0000","last_undegraded":"2024-04-25T16:27:02.900079+0000","last_fullsized":"2024-04-25T16:27:02.900079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584095+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:01.584095+0000","last_peered":"2024-04-25T16:27:01.584095+0000","last_clean":"2024-04-25T16:27:01.584095+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:01.584095+0000","last_undegraded":"2024-04-25T16:27:01.584095+0000","last_fullsized":"2024-04-25T16:27:01.584095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161817+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:02.161817+0000","last_peered":"2024-04-25T16:27:02.161817+0000","last_clean":"2024-04-25T16:27:02.161817+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:02.161817+0000","last_undegraded":"2024-04-25T16:27:02.161817+0000","last_fullsized":"2024-04-25T16:27:02.161817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900253+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:02.900253+0000","last_peered":"2024-04-25T16:27:02.900253+0000","last_clean":"2024-04-25T16:27:02.900253+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:02.900253+0000","last_undegraded":"2024-04-25T16:27:02.900253+0000","last_fullsized":"2024-04-25T16:27:02.900253+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589943+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:01.589943+0000","last_peered":"2024-04-25T16:27:01.589943+0000","last_clean":"2024-04-25T16:27:01.589943+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:01.589943+0000","last_undegraded":"2024-04-25T16:27:01.589943+0000","last_fullsized":"2024-04-25T16:27:01.589943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900575+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:02.900575+0000","last_peered":"2024-04-25T16:27:02.900575+0000","last_clean":"2024-04-25T16:27:02.900575+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:02.900575+0000","last_undegraded":"2024-04-25T16:27:02.900575+0000","last_fullsized":"2024-04-25T16:27:02.900575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900594+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:02.900594+0000","last_peered":"2024-04-25T16:27:02.900594+0000","last_clean":"2024-04-25T16:27:02.900594+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:02.900594+0000","last_undegraded":"2024-04-25T16:27:02.900594+0000","last_fullsized":"2024-04-25T16:27:02.900594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591187+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:01.591187+0000","last_peered":"2024-04-25T16:27:01.591187+0000","last_clean":"2024-04-25T16:27:01.591187+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:01.591187+0000","last_undegraded":"2024-04-25T16:27:01.591187+0000","last_fullsized":"2024-04-25T16:27:01.591187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161997+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:02.161997+0000","last_peered":"2024-04-25T16:27:02.161997+0000","last_clean":"2024-04-25T16:27:02.161997+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:02.161997+0000","last_undegraded":"2024-04-25T16:27:02.161997+0000","last_fullsized":"2024-04-25T16:27:02.161997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900648+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:02.900648+0000","last_peered":"2024-04-25T16:27:02.900648+0000","last_clean":"2024-04-25T16:27:02.900648+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:02.900648+0000","last_undegraded":"2024-04-25T16:27:02.900648+0000","last_fullsized":"2024-04-25T16:27:02.900648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901955+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:02.901955+0000","last_peered":"2024-04-25T16:27:02.901955+0000","last_clean":"2024-04-25T16:27:02.901955+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:02.901955+0000","last_undegraded":"2024-04-25T16:27:02.901955+0000","last_fullsized":"2024-04-25T16:27:02.901955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900856+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:02.900856+0000","last_peered":"2024-04-25T16:27:02.900856+0000","last_clean":"2024-04-25T16:27:02.900856+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:02.900856+0000","last_undegraded":"2024-04-25T16:27:02.900856+0000","last_fullsized":"2024-04-25T16:27:02.900856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901967+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:02.901967+0000","last_peered":"2024-04-25T16:27:02.901967+0000","last_clean":"2024-04-25T16:27:02.901967+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:02.901967+0000","last_undegraded":"2024-04-25T16:27:02.901967+0000","last_fullsized":"2024-04-25T16:27:02.901967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161498+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:02.161498+0000","last_peered":"2024-04-25T16:27:02.161498+0000","last_clean":"2024-04-25T16:27:02.161498+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:02.161498+0000","last_undegraded":"2024-04-25T16:27:02.161498+0000","last_fullsized":"2024-04-25T16:27:02.161498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900790+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:02.900790+0000","last_peered":"2024-04-25T16:27:02.900790+0000","last_clean":"2024-04-25T16:27:02.900790+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:02.900790+0000","last_undegraded":"2024-04-25T16:27:02.900790+0000","last_fullsized":"2024-04-25T16:27:02.900790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590401+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:01.590401+0000","last_peered":"2024-04-25T16:27:01.590401+0000","last_clean":"2024-04-25T16:27:01.590401+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:01.590401+0000","last_undegraded":"2024-04-25T16:27:01.590401+0000","last_fullsized":"2024-04-25T16:27:01.590401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589995+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:01.589995+0000","last_peered":"2024-04-25T16:27:01.589995+0000","last_clean":"2024-04-25T16:27:01.589995+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:01.589995+0000","last_undegraded":"2024-04-25T16:27:01.589995+0000","last_fullsized":"2024-04-25T16:27:01.589995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900320+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:02.900320+0000","last_peered":"2024-04-25T16:27:02.900320+0000","last_clean":"2024-04-25T16:27:02.900320+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:02.900320+0000","last_undegraded":"2024-04-25T16:27:02.900320+0000","last_fullsized":"2024-04-25T16:27:02.900320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900231+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:02.900231+0000","last_peered":"2024-04-25T16:27:02.900231+0000","last_clean":"2024-04-25T16:27:02.900231+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:02.900231+0000","last_undegraded":"2024-04-25T16:27:02.900231+0000","last_fullsized":"2024-04-25T16:27:02.900231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901358+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:02.901358+0000","last_peered":"2024-04-25T16:27:02.901358+0000","last_clean":"2024-04-25T16:27:02.901358+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:02.901358+0000","last_undegraded":"2024-04-25T16:27:02.901358+0000","last_fullsized":"2024-04-25T16:27:02.901358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589529+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:01.589529+0000","last_peered":"2024-04-25T16:27:01.589529+0000","last_clean":"2024-04-25T16:27:01.589529+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:01.589529+0000","last_undegraded":"2024-04-25T16:27:01.589529+0000","last_fullsized":"2024-04-25T16:27:01.589529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589509+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:01.589509+0000","last_peered":"2024-04-25T16:27:01.589509+0000","last_clean":"2024-04-25T16:27:01.589509+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:01.589509+0000","last_undegraded":"2024-04-25T16:27:01.589509+0000","last_fullsized":"2024-04-25T16:27:01.589509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584625+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:01.584625+0000","last_peered":"2024-04-25T16:27:01.584625+0000","last_clean":"2024-04-25T16:27:01.584625+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:01.584625+0000","last_undegraded":"2024-04-25T16:27:01.584625+0000","last_fullsized":"2024-04-25T16:27:01.584625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900487+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:02.900487+0000","last_peered":"2024-04-25T16:27:02.900487+0000","last_clean":"2024-04-25T16:27:02.900487+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:02.900487+0000","last_undegraded":"2024-04-25T16:27:02.900487+0000","last_fullsized":"2024-04-25T16:27:02.900487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161965+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:02.161965+0000","last_peered":"2024-04-25T16:27:02.161965+0000","last_clean":"2024-04-25T16:27:02.161965+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:02.161965+0000","last_undegraded":"2024-04-25T16:27:02.161965+0000","last_fullsized":"2024-04-25T16:27:02.161965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584525+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:01.584525+0000","last_peered":"2024-04-25T16:27:01.584525+0000","last_clean":"2024-04-25T16:27:01.584525+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:01.584525+0000","last_undegraded":"2024-04-25T16:27:01.584525+0000","last_fullsized":"2024-04-25T16:27:01.584525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902220+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:02.902220+0000","last_peered":"2024-04-25T16:27:02.902220+0000","last_clean":"2024-04-25T16:27:02.902220+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:02.902220+0000","last_undegraded":"2024-04-25T16:27:02.902220+0000","last_fullsized":"2024-04-25T16:27:02.902220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162131+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:02.162131+0000","last_peered":"2024-04-25T16:27:02.162131+0000","last_clean":"2024-04-25T16:27:02.162131+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:02.162131+0000","last_undegraded":"2024-04-25T16:27:02.162131+0000","last_fullsized":"2024-04-25T16:27:02.162131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900492+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:02.900492+0000","last_peered":"2024-04-25T16:27:02.900492+0000","last_clean":"2024-04-25T16:27:02.900492+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:02.900492+0000","last_undegraded":"2024-04-25T16:27:02.900492+0000","last_fullsized":"2024-04-25T16:27:02.900492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:07.253703+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:07.253703+0000","last_peered":"2024-04-25T16:27:07.253703+0000","last_clean":"2024-04-25T16:27:07.253703+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:07.253703+0000","last_undegraded":"2024-04-25T16:27:07.253703+0000","last_fullsized":"2024-04-25T16:27:07.253703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900859+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:02.900859+0000","last_peered":"2024-04-25T16:27:02.900859+0000","last_clean":"2024-04-25T16:27:02.900859+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:02.900859+0000","last_undegraded":"2024-04-25T16:27:02.900859+0000","last_fullsized":"2024-04-25T16:27:02.900859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590243+0000","last_change":"2024-04-25T16:26:56.570117+0000","last_active":"2024-04-25T16:27:01.590243+0000","last_peered":"2024-04-25T16:27:01.590243+0000","last_clean":"2024-04-25T16:27:01.590243+0000","last_became_active":"2024-04-25T16:26:56.569817+0000","last_became_peered":"2024-04-25T16:26:56.569817+0000","last_unstale":"2024-04-25T16:27:01.590243+0000","last_undegraded":"2024-04-25T16:27:01.590243+0000","last_fullsized":"2024-04-25T16:27:01.590243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:12.099635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900579+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:02.900579+0000","last_peered":"2024-04-25T16:27:02.900579+0000","last_clean":"2024-04-25T16:27:02.900579+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:02.900579+0000","last_undegraded":"2024-04-25T16:27:02.900579+0000","last_fullsized":"2024-04-25T16:27:02.900579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589933+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:01.589933+0000","last_peered":"2024-04-25T16:27:01.589933+0000","last_clean":"2024-04-25T16:27:01.589933+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:01.589933+0000","last_undegraded":"2024-04-25T16:27:01.589933+0000","last_fullsized":"2024-04-25T16:27:01.589933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162032+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:02.162032+0000","last_peered":"2024-04-25T16:27:02.162032+0000","last_clean":"2024-04-25T16:27:02.162032+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:02.162032+0000","last_undegraded":"2024-04-25T16:27:02.162032+0000","last_fullsized":"2024-04-25T16:27:02.162032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584281+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:01.584281+0000","last_peered":"2024-04-25T16:27:01.584281+0000","last_clean":"2024-04-25T16:27:01.584281+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:01.584281+0000","last_undegraded":"2024-04-25T16:27:01.584281+0000","last_fullsized":"2024-04-25T16:27:01.584281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161984+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:02.161984+0000","last_peered":"2024-04-25T16:27:02.161984+0000","last_clean":"2024-04-25T16:27:02.161984+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:02.161984+0000","last_undegraded":"2024-04-25T16:27:02.161984+0000","last_fullsized":"2024-04-25T16:27:02.161984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591106+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:01.591106+0000","last_peered":"2024-04-25T16:27:01.591106+0000","last_clean":"2024-04-25T16:27:01.591106+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:01.591106+0000","last_undegraded":"2024-04-25T16:27:01.591106+0000","last_fullsized":"2024-04-25T16:27:01.591106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584144+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:01.584144+0000","last_peered":"2024-04-25T16:27:01.584144+0000","last_clean":"2024-04-25T16:27:01.584144+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:01.584144+0000","last_undegraded":"2024-04-25T16:27:01.584144+0000","last_fullsized":"2024-04-25T16:27:01.584144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901211+0000","last_change":"2024-04-25T16:27:00.605248+0000","last_active":"2024-04-25T16:27:02.901211+0000","last_peered":"2024-04-25T16:27:02.901211+0000","last_clean":"2024-04-25T16:27:02.901211+0000","last_became_active":"2024-04-25T16:27:00.604746+0000","last_became_peered":"2024-04-25T16:27:00.604746+0000","last_unstale":"2024-04-25T16:27:02.901211+0000","last_undegraded":"2024-04-25T16:27:02.901211+0000","last_fullsized":"2024-04-25T16:27:02.901211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900313+0000","last_change":"2024-04-25T16:26:56.553599+0000","last_active":"2024-04-25T16:27:02.900313+0000","last_peered":"2024-04-25T16:27:02.900313+0000","last_clean":"2024-04-25T16:27:02.900313+0000","last_became_active":"2024-04-25T16:26:56.553304+0000","last_became_peered":"2024-04-25T16:26:56.553304+0000","last_unstale":"2024-04-25T16:27:02.900313+0000","last_undegraded":"2024-04-25T16:27:02.900313+0000","last_fullsized":"2024-04-25T16:27:02.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161615+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:02.161615+0000","last_peered":"2024-04-25T16:27:02.161615+0000","last_clean":"2024-04-25T16:27:02.161615+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:02.161615+0000","last_undegraded":"2024-04-25T16:27:02.161615+0000","last_fullsized":"2024-04-25T16:27:02.161615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591033+0000","last_change":"2024-04-25T16:27:00.587907+0000","last_active":"2024-04-25T16:27:01.591033+0000","last_peered":"2024-04-25T16:27:01.591033+0000","last_clean":"2024-04-25T16:27:01.591033+0000","last_became_active":"2024-04-25T16:27:00.587281+0000","last_became_peered":"2024-04-25T16:27:00.587281+0000","last_unstale":"2024-04-25T16:27:01.591033+0000","last_undegraded":"2024-04-25T16:27:01.591033+0000","last_fullsized":"2024-04-25T16:27:01.591033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584123+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:01.584123+0000","last_peered":"2024-04-25T16:27:01.584123+0000","last_clean":"2024-04-25T16:27:01.584123+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:01.584123+0000","last_undegraded":"2024-04-25T16:27:01.584123+0000","last_fullsized":"2024-04-25T16:27:01.584123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584701+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:01.584701+0000","last_peered":"2024-04-25T16:27:01.584701+0000","last_clean":"2024-04-25T16:27:01.584701+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:01.584701+0000","last_undegraded":"2024-04-25T16:27:01.584701+0000","last_fullsized":"2024-04-25T16:27:01.584701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591433+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:01.591433+0000","last_peered":"2024-04-25T16:27:01.591433+0000","last_clean":"2024-04-25T16:27:01.591433+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:01.591433+0000","last_undegraded":"2024-04-25T16:27:01.591433+0000","last_fullsized":"2024-04-25T16:27:01.591433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902190+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:02.902190+0000","last_peered":"2024-04-25T16:27:02.902190+0000","last_clean":"2024-04-25T16:27:02.902190+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:02.902190+0000","last_undegraded":"2024-04-25T16:27:02.902190+0000","last_fullsized":"2024-04-25T16:27:02.902190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900327+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:02.900327+0000","last_peered":"2024-04-25T16:27:02.900327+0000","last_clean":"2024-04-25T16:27:02.900327+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:02.900327+0000","last_undegraded":"2024-04-25T16:27:02.900327+0000","last_fullsized":"2024-04-25T16:27:02.900327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584664+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:01.584664+0000","last_peered":"2024-04-25T16:27:01.584664+0000","last_clean":"2024-04-25T16:27:01.584664+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:01.584664+0000","last_undegraded":"2024-04-25T16:27:01.584664+0000","last_fullsized":"2024-04-25T16:27:01.584664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589742+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:01.589742+0000","last_peered":"2024-04-25T16:27:01.589742+0000","last_clean":"2024-04-25T16:27:01.589742+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:01.589742+0000","last_undegraded":"2024-04-25T16:27:01.589742+0000","last_fullsized":"2024-04-25T16:27:01.589742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584547+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:01.584547+0000","last_peered":"2024-04-25T16:27:01.584547+0000","last_clean":"2024-04-25T16:27:01.584547+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:01.584547+0000","last_undegraded":"2024-04-25T16:27:01.584547+0000","last_fullsized":"2024-04-25T16:27:01.584547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589145+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:01.589145+0000","last_peered":"2024-04-25T16:27:01.589145+0000","last_clean":"2024-04-25T16:27:01.589145+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:01.589145+0000","last_undegraded":"2024-04-25T16:27:01.589145+0000","last_fullsized":"2024-04-25T16:27:01.589145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.899980+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:02.899980+0000","last_peered":"2024-04-25T16:27:02.899980+0000","last_clean":"2024-04-25T16:27:02.899980+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:02.899980+0000","last_undegraded":"2024-04-25T16:27:02.899980+0000","last_fullsized":"2024-04-25T16:27:02.899980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161855+0000","last_change":"2024-04-25T16:26:56.558625+0000","last_active":"2024-04-25T16:27:02.161855+0000","last_peered":"2024-04-25T16:27:02.161855+0000","last_clean":"2024-04-25T16:27:02.161855+0000","last_became_active":"2024-04-25T16:26:56.558322+0000","last_became_peered":"2024-04-25T16:26:56.558322+0000","last_unstale":"2024-04-25T16:27:02.161855+0000","last_undegraded":"2024-04-25T16:27:02.161855+0000","last_fullsized":"2024-04-25T16:27:02.161855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902073+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:02.902073+0000","last_peered":"2024-04-25T16:27:02.902073+0000","last_clean":"2024-04-25T16:27:02.902073+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:02.902073+0000","last_undegraded":"2024-04-25T16:27:02.902073+0000","last_fullsized":"2024-04-25T16:27:02.902073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584798+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:01.584798+0000","last_peered":"2024-04-25T16:27:01.584798+0000","last_clean":"2024-04-25T16:27:01.584798+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:01.584798+0000","last_undegraded":"2024-04-25T16:27:01.584798+0000","last_fullsized":"2024-04-25T16:27:01.584798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590378+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:01.590378+0000","last_peered":"2024-04-25T16:27:01.590378+0000","last_clean":"2024-04-25T16:27:01.590378+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:01.590378+0000","last_undegraded":"2024-04-25T16:27:01.590378+0000","last_fullsized":"2024-04-25T16:27:01.590378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584670+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:01.584670+0000","last_peered":"2024-04-25T16:27:01.584670+0000","last_clean":"2024-04-25T16:27:01.584670+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:01.584670+0000","last_undegraded":"2024-04-25T16:27:01.584670+0000","last_fullsized":"2024-04-25T16:27:01.584670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161594+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:02.161594+0000","last_peered":"2024-04-25T16:27:02.161594+0000","last_clean":"2024-04-25T16:27:02.161594+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:02.161594+0000","last_undegraded":"2024-04-25T16:27:02.161594+0000","last_fullsized":"2024-04-25T16:27:02.161594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584586+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:01.584586+0000","last_peered":"2024-04-25T16:27:01.584586+0000","last_clean":"2024-04-25T16:27:01.584586+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:01.584586+0000","last_undegraded":"2024-04-25T16:27:01.584586+0000","last_fullsized":"2024-04-25T16:27:01.584586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161810+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:02.161810+0000","last_peered":"2024-04-25T16:27:02.161810+0000","last_clean":"2024-04-25T16:27:02.161810+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:02.161810+0000","last_undegraded":"2024-04-25T16:27:02.161810+0000","last_fullsized":"2024-04-25T16:27:02.161810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901772+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:02.901772+0000","last_peered":"2024-04-25T16:27:02.901772+0000","last_clean":"2024-04-25T16:27:02.901772+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:02.901772+0000","last_undegraded":"2024-04-25T16:27:02.901772+0000","last_fullsized":"2024-04-25T16:27:02.901772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589540+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:01.589540+0000","last_peered":"2024-04-25T16:27:01.589540+0000","last_clean":"2024-04-25T16:27:01.589540+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:01.589540+0000","last_undegraded":"2024-04-25T16:27:01.589540+0000","last_fullsized":"2024-04-25T16:27:01.589540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900759+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:02.900759+0000","last_peered":"2024-04-25T16:27:02.900759+0000","last_clean":"2024-04-25T16:27:02.900759+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:02.900759+0000","last_undegraded":"2024-04-25T16:27:02.900759+0000","last_fullsized":"2024-04-25T16:27:02.900759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901860+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:02.901860+0000","last_peered":"2024-04-25T16:27:02.901860+0000","last_clean":"2024-04-25T16:27:02.901860+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:02.901860+0000","last_undegraded":"2024-04-25T16:27:02.901860+0000","last_fullsized":"2024-04-25T16:27:02.901860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162241+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:02.162241+0000","last_peered":"2024-04-25T16:27:02.162241+0000","last_clean":"2024-04-25T16:27:02.162241+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:02.162241+0000","last_undegraded":"2024-04-25T16:27:02.162241+0000","last_fullsized":"2024-04-25T16:27:02.162241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161412+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:02.161412+0000","last_peered":"2024-04-25T16:27:02.161412+0000","last_clean":"2024-04-25T16:27:02.161412+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:02.161412+0000","last_undegraded":"2024-04-25T16:27:02.161412+0000","last_fullsized":"2024-04-25T16:27:02.161412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900423+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:02.900423+0000","last_peered":"2024-04-25T16:27:02.900423+0000","last_clean":"2024-04-25T16:27:02.900423+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:02.900423+0000","last_undegraded":"2024-04-25T16:27:02.900423+0000","last_fullsized":"2024-04-25T16:27:02.900423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584351+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:01.584351+0000","last_peered":"2024-04-25T16:27:01.584351+0000","last_clean":"2024-04-25T16:27:01.584351+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:01.584351+0000","last_undegraded":"2024-04-25T16:27:01.584351+0000","last_fullsized":"2024-04-25T16:27:01.584351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590124+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:01.590124+0000","last_peered":"2024-04-25T16:27:01.590124+0000","last_clean":"2024-04-25T16:27:01.590124+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:01.590124+0000","last_undegraded":"2024-04-25T16:27:01.590124+0000","last_fullsized":"2024-04-25T16:27:01.590124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590011+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:01.590011+0000","last_peered":"2024-04-25T16:27:01.590011+0000","last_clean":"2024-04-25T16:27:01.590011+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:01.590011+0000","last_undegraded":"2024-04-25T16:27:01.590011+0000","last_fullsized":"2024-04-25T16:27:01.590011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590256+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:01.590256+0000","last_peered":"2024-04-25T16:27:01.590256+0000","last_clean":"2024-04-25T16:27:01.590256+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:01.590256+0000","last_undegraded":"2024-04-25T16:27:01.590256+0000","last_fullsized":"2024-04-25T16:27:01.590256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900847+0000","last_change":"2024-04-25T16:26:56.572739+0000","last_active":"2024-04-25T16:27:02.900847+0000","last_peered":"2024-04-25T16:27:02.900847+0000","last_clean":"2024-04-25T16:27:02.900847+0000","last_became_active":"2024-04-25T16:26:56.571251+0000","last_became_peered":"2024-04-25T16:26:56.571251+0000","last_unstale":"2024-04-25T16:27:02.900847+0000","last_undegraded":"2024-04-25T16:27:02.900847+0000","last_fullsized":"2024-04-25T16:27:02.900847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584005+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:01.584005+0000","last_peered":"2024-04-25T16:27:01.584005+0000","last_clean":"2024-04-25T16:27:01.584005+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:01.584005+0000","last_undegraded":"2024-04-25T16:27:01.584005+0000","last_fullsized":"2024-04-25T16:27:01.584005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900188+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:02.900188+0000","last_peered":"2024-04-25T16:27:02.900188+0000","last_clean":"2024-04-25T16:27:02.900188+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:02.900188+0000","last_undegraded":"2024-04-25T16:27:02.900188+0000","last_fullsized":"2024-04-25T16:27:02.900188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161603+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:02.161603+0000","last_peered":"2024-04-25T16:27:02.161603+0000","last_clean":"2024-04-25T16:27:02.161603+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:02.161603+0000","last_undegraded":"2024-04-25T16:27:02.161603+0000","last_fullsized":"2024-04-25T16:27:02.161603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161799+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:02.161799+0000","last_peered":"2024-04-25T16:27:02.161799+0000","last_clean":"2024-04-25T16:27:02.161799+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:02.161799+0000","last_undegraded":"2024-04-25T16:27:02.161799+0000","last_fullsized":"2024-04-25T16:27:02.161799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900832+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:02.900832+0000","last_peered":"2024-04-25T16:27:02.900832+0000","last_clean":"2024-04-25T16:27:02.900832+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:02.900832+0000","last_undegraded":"2024-04-25T16:27:02.900832+0000","last_fullsized":"2024-04-25T16:27:02.900832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901018+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:02.901018+0000","last_peered":"2024-04-25T16:27:02.901018+0000","last_clean":"2024-04-25T16:27:02.901018+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:02.901018+0000","last_undegraded":"2024-04-25T16:27:02.901018+0000","last_fullsized":"2024-04-25T16:27:02.901018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900715+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:02.900715+0000","last_peered":"2024-04-25T16:27:02.900715+0000","last_clean":"2024-04-25T16:27:02.900715+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:02.900715+0000","last_undegraded":"2024-04-25T16:27:02.900715+0000","last_fullsized":"2024-04-25T16:27:02.900715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":103933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":694813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:27.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:27.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.474+0000 7f2ad5030640 1 Processor -- start 2024-04-25T16:27:27.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.474+0000 7f2ad5030640 1 -- start start 2024-04-25T16:27:27.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ad5030640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ad008d4c0 0x7f2ad008d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ad5030640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ad00b7fe0 0x7f2ad00b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ad5030640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad008de10 con 0x7f2ad00b46b0 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ad5030640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad00b51d0 con 0x7f2ad008d4c0 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ad5030640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad00b6f80 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2aced9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ad008d4c0 0x7f2ad008d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ac7fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2ad00b46b0 0x7f2ad00b4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56316/0 (socket says 172.21.15.94:56316) 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2aced9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ad008d4c0 0x7f2ad008d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56232/0 (socket says 172.21.15.94:56232) 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ac7fff640 1 -- 172.21.15.94:0/1319405853 learned_addr learned my addr 172.21.15.94:0/1319405853 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2acf59e640 1 --2- 172.21.15.94:0/1319405853 >> v2:172.21.15.138:3300/0 conn(0x7f2ad00b7fe0 0x7f2ad00b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2aced9d640 1 -- 172.21.15.94:0/1319405853 >> v2:172.21.15.138:3300/0 conn(0x7f2ad00b7fe0 msgr2=0x7f2ad00b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2aced9d640 1 --2- 172.21.15.94:0/1319405853 >> v2:172.21.15.138:3300/0 conn(0x7f2ad00b7fe0 0x7f2ad00b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2aced9d640 1 -- 172.21.15.94:0/1319405853 >> v1:172.21.15.94:6789/0 conn(0x7f2ad00b46b0 legacy=0x7f2ad00b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2ace59c640 1 -- 172.21.15.94:0/1319405853 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3382340927 0 0) 0x7f2ab8001110 con 0x7f2ad00b46b0 2024-04-25T16:27:27.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.476+0000 7f2aced9d640 1 -- 172.21.15.94:0/1319405853 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad00b7220 con 0x7f2ad008d4c0 2024-04-25T16:27:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.477+0000 7f2aced9d640 1 --2- 172.21.15.94:0/1319405853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad008d4c0 0x7f2ad008d8d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ad008e900 tx=0x7f2ac002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703ec2d60fd98f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.477+0000 7f2ace59c640 1 -- 172.21.15.94:0/1319405853 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2ad008d4c0 2024-04-25T16:27:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.477+0000 7f2ace59c640 1 -- 172.21.15.94:0/1319405853 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0032440 con 0x7f2ad008d4c0 2024-04-25T16:27:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.477+0000 7f2ace59c640 1 -- 172.21.15.94:0/1319405853 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0032810 con 0x7f2ad008d4c0 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 -- 172.21.15.94:0/1319405853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad008d4c0 msgr2=0x7f2ad008d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 --2- 172.21.15.94:0/1319405853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad008d4c0 0x7f2ad008d8d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ad008e900 tx=0x7f2ac002a1a0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 -- 172.21.15.94:0/1319405853 shutdown_connections 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 --2- 172.21.15.94:0/1319405853 >> v2:172.21.15.138:3300/0 conn(0x7f2ad00b7fe0 0x7f2ad00b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 --2- 172.21.15.94:0/1319405853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad008d4c0 0x7f2ad008d8d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 -- 172.21.15.94:0/1319405853 >> 172.21.15.94:0/1319405853 conn(0x7f2ad00d4090 msgr2=0x7f2ad00d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 -- 172.21.15.94:0/1319405853 shutdown_connections 2024-04-25T16:27:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.478+0000 7f2ad5030640 1 -- 172.21.15.94:0/1319405853 wait complete. 2024-04-25T16:27:27.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.480+0000 7f2ad5030640 1 Processor -- start 2024-04-25T16:27:27.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.480+0000 7f2ad5030640 1 -- start start 2024-04-25T16:27:27.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2ad5030640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ad008d4c0 0x7f2ad006fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2ad5030640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad00b46b0 0x7f2ad00b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2ad5030640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 0x7f2ad00b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2ad5030640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00b7ac0 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2ad5030640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00b77e0 con 0x7f2ad00b46b0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2ad5030640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00b3870 con 0x7f2ad008d4c0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2acf59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 0x7f2ad00b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2acf59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 0x7f2ad00b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56244/0 (socket says 172.21.15.94:56244) 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.481+0000 7f2acf59e640 1 -- 172.21.15.94:0/323984428 learned_addr learned my addr 172.21.15.94:0/323984428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2ac7fff640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad00b46b0 0x7f2ad00b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2aced9d640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ad008d4c0 0x7f2ad006fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2acf59e640 1 -- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ad008d4c0 msgr2=0x7f2ad006fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2acf59e640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ad008d4c0 0x7f2ad006fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2acf59e640 1 -- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad00b46b0 msgr2=0x7f2ad00b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2acf59e640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad00b46b0 0x7f2ad00b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2acf59e640 1 -- 172.21.15.94:0/323984428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad006e0a0 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2aced9d640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ad008d4c0 0x7f2ad006fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2acf59e640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 0x7f2ad00b2250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007e90 tx=0x7f2ac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8014420 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac8019070 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad01f6950 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8014800 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.482+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad01fac60 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.483+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad015e030 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.483+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ac80149a0 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.484+0000 7f2ac77fe640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ab00744b0 0x7f2ab00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.484+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2ac8095d00 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.484+0000 7f2aced9d640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ab00744b0 0x7f2ab00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.484+0000 7f2aced9d640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ab00744b0 0x7f2ab00769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0034000 tx=0x7f2ac002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.489+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ac805fb90 con 0x7f2ad00b7fe0 2024-04-25T16:27:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.666+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ad006a180 con 0x7f2ab00744b0 2024-04-25T16:27:27.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.673+0000 7f2ac77fe640 1 -- 172.21.15.94:0/323984428 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332520 (secure 0 0 0) 0x7f2ad006a180 con 0x7f2ab00744b0 2024-04-25T16:27:27.677 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ab00744b0 msgr2=0x7f2ab00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ab00744b0 0x7f2ab00769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0034000 tx=0x7f2ac002e000 comp rx=0 tx=0).stop 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 msgr2=0x7f2ad00b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 0x7f2ad00b2250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007e90 tx=0x7f2ac800e4b0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 shutdown_connections 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2ab00744b0 0x7f2ab00769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad00b7fe0 0x7f2ad00b2250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad00b46b0 0x7f2ad00b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 --2- 172.21.15.94:0/323984428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ad008d4c0 0x7f2ad006fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 >> 172.21.15.94:0/323984428 conn(0x7f2ad00d4090 msgr2=0x7f2ad00b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 shutdown_connections 2024-04-25T16:27:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.677+0000 7f2ad5030640 1 -- 172.21.15.94:0/323984428 wait complete. 2024-04-25T16:27:27.688 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-25T16:27:27.304620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236072,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738648,"statfs":{"total":773094113280,"available":772852375552,"internally_reserved":0,"allocated":3350528,"data_stored":2020472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003276"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901527+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:02.901527+0000","last_peered":"2024-04-25T16:27:02.901527+0000","last_clean":"2024-04-25T16:27:02.901527+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:02.901527+0000","last_undegraded":"2024-04-25T16:27:02.901527+0000","last_fullsized":"2024-04-25T16:27:02.901527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584607+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:01.584607+0000","last_peered":"2024-04-25T16:27:01.584607+0000","last_clean":"2024-04-25T16:27:01.584607+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:01.584607+0000","last_undegraded":"2024-04-25T16:27:01.584607+0000","last_fullsized":"2024-04-25T16:27:01.584607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584101+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:01.584101+0000","last_peered":"2024-04-25T16:27:01.584101+0000","last_clean":"2024-04-25T16:27:01.584101+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:01.584101+0000","last_undegraded":"2024-04-25T16:27:01.584101+0000","last_fullsized":"2024-04-25T16:27:01.584101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584845+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:01.584845+0000","last_peered":"2024-04-25T16:27:01.584845+0000","last_clean":"2024-04-25T16:27:01.584845+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:01.584845+0000","last_undegraded":"2024-04-25T16:27:01.584845+0000","last_fullsized":"2024-04-25T16:27:01.584845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901225+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:02.901225+0000","last_peered":"2024-04-25T16:27:02.901225+0000","last_clean":"2024-04-25T16:27:02.901225+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:02.901225+0000","last_undegraded":"2024-04-25T16:27:02.901225+0000","last_fullsized":"2024-04-25T16:27:02.901225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589204+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:01.589204+0000","last_peered":"2024-04-25T16:27:01.589204+0000","last_clean":"2024-04-25T16:27:01.589204+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:01.589204+0000","last_undegraded":"2024-04-25T16:27:01.589204+0000","last_fullsized":"2024-04-25T16:27:01.589204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900752+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:02.900752+0000","last_peered":"2024-04-25T16:27:02.900752+0000","last_clean":"2024-04-25T16:27:02.900752+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:02.900752+0000","last_undegraded":"2024-04-25T16:27:02.900752+0000","last_fullsized":"2024-04-25T16:27:02.900752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161964+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:02.161964+0000","last_peered":"2024-04-25T16:27:02.161964+0000","last_clean":"2024-04-25T16:27:02.161964+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:02.161964+0000","last_undegraded":"2024-04-25T16:27:02.161964+0000","last_fullsized":"2024-04-25T16:27:02.161964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589641+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:01.589641+0000","last_peered":"2024-04-25T16:27:01.589641+0000","last_clean":"2024-04-25T16:27:01.589641+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:01.589641+0000","last_undegraded":"2024-04-25T16:27:01.589641+0000","last_fullsized":"2024-04-25T16:27:01.589641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900437+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:02.900437+0000","last_peered":"2024-04-25T16:27:02.900437+0000","last_clean":"2024-04-25T16:27:02.900437+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:02.900437+0000","last_undegraded":"2024-04-25T16:27:02.900437+0000","last_fullsized":"2024-04-25T16:27:02.900437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162051+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:02.162051+0000","last_peered":"2024-04-25T16:27:02.162051+0000","last_clean":"2024-04-25T16:27:02.162051+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:02.162051+0000","last_undegraded":"2024-04-25T16:27:02.162051+0000","last_fullsized":"2024-04-25T16:27:02.162051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589525+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:01.589525+0000","last_peered":"2024-04-25T16:27:01.589525+0000","last_clean":"2024-04-25T16:27:01.589525+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:01.589525+0000","last_undegraded":"2024-04-25T16:27:01.589525+0000","last_fullsized":"2024-04-25T16:27:01.589525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591279+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:01.591279+0000","last_peered":"2024-04-25T16:27:01.591279+0000","last_clean":"2024-04-25T16:27:01.591279+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:01.591279+0000","last_undegraded":"2024-04-25T16:27:01.591279+0000","last_fullsized":"2024-04-25T16:27:01.591279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900534+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:02.900534+0000","last_peered":"2024-04-25T16:27:02.900534+0000","last_clean":"2024-04-25T16:27:02.900534+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:02.900534+0000","last_undegraded":"2024-04-25T16:27:02.900534+0000","last_fullsized":"2024-04-25T16:27:02.900534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591334+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:01.591334+0000","last_peered":"2024-04-25T16:27:01.591334+0000","last_clean":"2024-04-25T16:27:01.591334+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:01.591334+0000","last_undegraded":"2024-04-25T16:27:01.591334+0000","last_fullsized":"2024-04-25T16:27:01.591334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584779+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:01.584779+0000","last_peered":"2024-04-25T16:27:01.584779+0000","last_clean":"2024-04-25T16:27:01.584779+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:01.584779+0000","last_undegraded":"2024-04-25T16:27:01.584779+0000","last_fullsized":"2024-04-25T16:27:01.584779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589901+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:01.589901+0000","last_peered":"2024-04-25T16:27:01.589901+0000","last_clean":"2024-04-25T16:27:01.589901+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:01.589901+0000","last_undegraded":"2024-04-25T16:27:01.589901+0000","last_fullsized":"2024-04-25T16:27:01.589901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584240+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:01.584240+0000","last_peered":"2024-04-25T16:27:01.584240+0000","last_clean":"2024-04-25T16:27:01.584240+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:01.584240+0000","last_undegraded":"2024-04-25T16:27:01.584240+0000","last_fullsized":"2024-04-25T16:27:01.584240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900529+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:02.900529+0000","last_peered":"2024-04-25T16:27:02.900529+0000","last_clean":"2024-04-25T16:27:02.900529+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:02.900529+0000","last_undegraded":"2024-04-25T16:27:02.900529+0000","last_fullsized":"2024-04-25T16:27:02.900529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901233+0000","last_change":"2024-04-25T16:27:00.605424+0000","last_active":"2024-04-25T16:27:02.901233+0000","last_peered":"2024-04-25T16:27:02.901233+0000","last_clean":"2024-04-25T16:27:02.901233+0000","last_became_active":"2024-04-25T16:27:00.604972+0000","last_became_peered":"2024-04-25T16:27:00.604972+0000","last_unstale":"2024-04-25T16:27:02.901233+0000","last_undegraded":"2024-04-25T16:27:02.901233+0000","last_fullsized":"2024-04-25T16:27:02.901233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590785+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:01.590785+0000","last_peered":"2024-04-25T16:27:01.590785+0000","last_clean":"2024-04-25T16:27:01.590785+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:01.590785+0000","last_undegraded":"2024-04-25T16:27:01.590785+0000","last_fullsized":"2024-04-25T16:27:01.590785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900076+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:02.900076+0000","last_peered":"2024-04-25T16:27:02.900076+0000","last_clean":"2024-04-25T16:27:02.900076+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:02.900076+0000","last_undegraded":"2024-04-25T16:27:02.900076+0000","last_fullsized":"2024-04-25T16:27:02.900076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591342+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:01.591342+0000","last_peered":"2024-04-25T16:27:01.591342+0000","last_clean":"2024-04-25T16:27:01.591342+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:01.591342+0000","last_undegraded":"2024-04-25T16:27:01.591342+0000","last_fullsized":"2024-04-25T16:27:01.591342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901614+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:02.901614+0000","last_peered":"2024-04-25T16:27:02.901614+0000","last_clean":"2024-04-25T16:27:02.901614+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:02.901614+0000","last_undegraded":"2024-04-25T16:27:02.901614+0000","last_fullsized":"2024-04-25T16:27:02.901614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591218+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:01.591218+0000","last_peered":"2024-04-25T16:27:01.591218+0000","last_clean":"2024-04-25T16:27:01.591218+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:01.591218+0000","last_undegraded":"2024-04-25T16:27:01.591218+0000","last_fullsized":"2024-04-25T16:27:01.591218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590686+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:01.590686+0000","last_peered":"2024-04-25T16:27:01.590686+0000","last_clean":"2024-04-25T16:27:01.590686+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:01.590686+0000","last_undegraded":"2024-04-25T16:27:01.590686+0000","last_fullsized":"2024-04-25T16:27:01.590686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584723+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:01.584723+0000","last_peered":"2024-04-25T16:27:01.584723+0000","last_clean":"2024-04-25T16:27:01.584723+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:01.584723+0000","last_undegraded":"2024-04-25T16:27:01.584723+0000","last_fullsized":"2024-04-25T16:27:01.584723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901145+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:02.901145+0000","last_peered":"2024-04-25T16:27:02.901145+0000","last_clean":"2024-04-25T16:27:02.901145+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:02.901145+0000","last_undegraded":"2024-04-25T16:27:02.901145+0000","last_fullsized":"2024-04-25T16:27:02.901145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.583975+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:01.583975+0000","last_peered":"2024-04-25T16:27:01.583975+0000","last_clean":"2024-04-25T16:27:01.583975+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:01.583975+0000","last_undegraded":"2024-04-25T16:27:01.583975+0000","last_fullsized":"2024-04-25T16:27:01.583975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900898+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:02.900898+0000","last_peered":"2024-04-25T16:27:02.900898+0000","last_clean":"2024-04-25T16:27:02.900898+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:02.900898+0000","last_undegraded":"2024-04-25T16:27:02.900898+0000","last_fullsized":"2024-04-25T16:27:02.900898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900860+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:02.900860+0000","last_peered":"2024-04-25T16:27:02.900860+0000","last_clean":"2024-04-25T16:27:02.900860+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:02.900860+0000","last_undegraded":"2024-04-25T16:27:02.900860+0000","last_fullsized":"2024-04-25T16:27:02.900860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901651+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:02.901651+0000","last_peered":"2024-04-25T16:27:02.901651+0000","last_clean":"2024-04-25T16:27:02.901651+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:02.901651+0000","last_undegraded":"2024-04-25T16:27:02.901651+0000","last_fullsized":"2024-04-25T16:27:02.901651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162233+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:02.162233+0000","last_peered":"2024-04-25T16:27:02.162233+0000","last_clean":"2024-04-25T16:27:02.162233+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:02.162233+0000","last_undegraded":"2024-04-25T16:27:02.162233+0000","last_fullsized":"2024-04-25T16:27:02.162233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901021+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:02.901021+0000","last_peered":"2024-04-25T16:27:02.901021+0000","last_clean":"2024-04-25T16:27:02.901021+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:02.901021+0000","last_undegraded":"2024-04-25T16:27:02.901021+0000","last_fullsized":"2024-04-25T16:27:02.901021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900663+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:02.900663+0000","last_peered":"2024-04-25T16:27:02.900663+0000","last_clean":"2024-04-25T16:27:02.900663+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:02.900663+0000","last_undegraded":"2024-04-25T16:27:02.900663+0000","last_fullsized":"2024-04-25T16:27:02.900663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590164+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:01.590164+0000","last_peered":"2024-04-25T16:27:01.590164+0000","last_clean":"2024-04-25T16:27:01.590164+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:01.590164+0000","last_undegraded":"2024-04-25T16:27:01.590164+0000","last_fullsized":"2024-04-25T16:27:01.590164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162067+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:02.162067+0000","last_peered":"2024-04-25T16:27:02.162067+0000","last_clean":"2024-04-25T16:27:02.162067+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:02.162067+0000","last_undegraded":"2024-04-25T16:27:02.162067+0000","last_fullsized":"2024-04-25T16:27:02.162067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900915+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:02.900915+0000","last_peered":"2024-04-25T16:27:02.900915+0000","last_clean":"2024-04-25T16:27:02.900915+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:02.900915+0000","last_undegraded":"2024-04-25T16:27:02.900915+0000","last_fullsized":"2024-04-25T16:27:02.900915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900165+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:02.900165+0000","last_peered":"2024-04-25T16:27:02.900165+0000","last_clean":"2024-04-25T16:27:02.900165+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:02.900165+0000","last_undegraded":"2024-04-25T16:27:02.900165+0000","last_fullsized":"2024-04-25T16:27:02.900165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900098+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:02.900098+0000","last_peered":"2024-04-25T16:27:02.900098+0000","last_clean":"2024-04-25T16:27:02.900098+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:02.900098+0000","last_undegraded":"2024-04-25T16:27:02.900098+0000","last_fullsized":"2024-04-25T16:27:02.900098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901408+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:02.901408+0000","last_peered":"2024-04-25T16:27:02.901408+0000","last_clean":"2024-04-25T16:27:02.901408+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:02.901408+0000","last_undegraded":"2024-04-25T16:27:02.901408+0000","last_fullsized":"2024-04-25T16:27:02.901408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901450+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:02.901450+0000","last_peered":"2024-04-25T16:27:02.901450+0000","last_clean":"2024-04-25T16:27:02.901450+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:02.901450+0000","last_undegraded":"2024-04-25T16:27:02.901450+0000","last_fullsized":"2024-04-25T16:27:02.901450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900079+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:02.900079+0000","last_peered":"2024-04-25T16:27:02.900079+0000","last_clean":"2024-04-25T16:27:02.900079+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:02.900079+0000","last_undegraded":"2024-04-25T16:27:02.900079+0000","last_fullsized":"2024-04-25T16:27:02.900079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584095+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:01.584095+0000","last_peered":"2024-04-25T16:27:01.584095+0000","last_clean":"2024-04-25T16:27:01.584095+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:01.584095+0000","last_undegraded":"2024-04-25T16:27:01.584095+0000","last_fullsized":"2024-04-25T16:27:01.584095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161817+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:02.161817+0000","last_peered":"2024-04-25T16:27:02.161817+0000","last_clean":"2024-04-25T16:27:02.161817+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:02.161817+0000","last_undegraded":"2024-04-25T16:27:02.161817+0000","last_fullsized":"2024-04-25T16:27:02.161817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900253+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:02.900253+0000","last_peered":"2024-04-25T16:27:02.900253+0000","last_clean":"2024-04-25T16:27:02.900253+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:02.900253+0000","last_undegraded":"2024-04-25T16:27:02.900253+0000","last_fullsized":"2024-04-25T16:27:02.900253+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589943+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:01.589943+0000","last_peered":"2024-04-25T16:27:01.589943+0000","last_clean":"2024-04-25T16:27:01.589943+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:01.589943+0000","last_undegraded":"2024-04-25T16:27:01.589943+0000","last_fullsized":"2024-04-25T16:27:01.589943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900575+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:02.900575+0000","last_peered":"2024-04-25T16:27:02.900575+0000","last_clean":"2024-04-25T16:27:02.900575+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:02.900575+0000","last_undegraded":"2024-04-25T16:27:02.900575+0000","last_fullsized":"2024-04-25T16:27:02.900575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900594+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:02.900594+0000","last_peered":"2024-04-25T16:27:02.900594+0000","last_clean":"2024-04-25T16:27:02.900594+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:02.900594+0000","last_undegraded":"2024-04-25T16:27:02.900594+0000","last_fullsized":"2024-04-25T16:27:02.900594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591187+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:01.591187+0000","last_peered":"2024-04-25T16:27:01.591187+0000","last_clean":"2024-04-25T16:27:01.591187+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:01.591187+0000","last_undegraded":"2024-04-25T16:27:01.591187+0000","last_fullsized":"2024-04-25T16:27:01.591187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161997+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:02.161997+0000","last_peered":"2024-04-25T16:27:02.161997+0000","last_clean":"2024-04-25T16:27:02.161997+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:02.161997+0000","last_undegraded":"2024-04-25T16:27:02.161997+0000","last_fullsized":"2024-04-25T16:27:02.161997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900648+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:02.900648+0000","last_peered":"2024-04-25T16:27:02.900648+0000","last_clean":"2024-04-25T16:27:02.900648+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:02.900648+0000","last_undegraded":"2024-04-25T16:27:02.900648+0000","last_fullsized":"2024-04-25T16:27:02.900648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901955+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:02.901955+0000","last_peered":"2024-04-25T16:27:02.901955+0000","last_clean":"2024-04-25T16:27:02.901955+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:02.901955+0000","last_undegraded":"2024-04-25T16:27:02.901955+0000","last_fullsized":"2024-04-25T16:27:02.901955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900856+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:02.900856+0000","last_peered":"2024-04-25T16:27:02.900856+0000","last_clean":"2024-04-25T16:27:02.900856+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:02.900856+0000","last_undegraded":"2024-04-25T16:27:02.900856+0000","last_fullsized":"2024-04-25T16:27:02.900856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901967+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:02.901967+0000","last_peered":"2024-04-25T16:27:02.901967+0000","last_clean":"2024-04-25T16:27:02.901967+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:02.901967+0000","last_undegraded":"2024-04-25T16:27:02.901967+0000","last_fullsized":"2024-04-25T16:27:02.901967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161498+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:02.161498+0000","last_peered":"2024-04-25T16:27:02.161498+0000","last_clean":"2024-04-25T16:27:02.161498+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:02.161498+0000","last_undegraded":"2024-04-25T16:27:02.161498+0000","last_fullsized":"2024-04-25T16:27:02.161498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900790+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:02.900790+0000","last_peered":"2024-04-25T16:27:02.900790+0000","last_clean":"2024-04-25T16:27:02.900790+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:02.900790+0000","last_undegraded":"2024-04-25T16:27:02.900790+0000","last_fullsized":"2024-04-25T16:27:02.900790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590401+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:01.590401+0000","last_peered":"2024-04-25T16:27:01.590401+0000","last_clean":"2024-04-25T16:27:01.590401+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:01.590401+0000","last_undegraded":"2024-04-25T16:27:01.590401+0000","last_fullsized":"2024-04-25T16:27:01.590401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589995+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:01.589995+0000","last_peered":"2024-04-25T16:27:01.589995+0000","last_clean":"2024-04-25T16:27:01.589995+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:01.589995+0000","last_undegraded":"2024-04-25T16:27:01.589995+0000","last_fullsized":"2024-04-25T16:27:01.589995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900320+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:02.900320+0000","last_peered":"2024-04-25T16:27:02.900320+0000","last_clean":"2024-04-25T16:27:02.900320+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:02.900320+0000","last_undegraded":"2024-04-25T16:27:02.900320+0000","last_fullsized":"2024-04-25T16:27:02.900320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900231+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:02.900231+0000","last_peered":"2024-04-25T16:27:02.900231+0000","last_clean":"2024-04-25T16:27:02.900231+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:02.900231+0000","last_undegraded":"2024-04-25T16:27:02.900231+0000","last_fullsized":"2024-04-25T16:27:02.900231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901358+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:02.901358+0000","last_peered":"2024-04-25T16:27:02.901358+0000","last_clean":"2024-04-25T16:27:02.901358+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:02.901358+0000","last_undegraded":"2024-04-25T16:27:02.901358+0000","last_fullsized":"2024-04-25T16:27:02.901358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589529+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:01.589529+0000","last_peered":"2024-04-25T16:27:01.589529+0000","last_clean":"2024-04-25T16:27:01.589529+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:01.589529+0000","last_undegraded":"2024-04-25T16:27:01.589529+0000","last_fullsized":"2024-04-25T16:27:01.589529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589509+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:01.589509+0000","last_peered":"2024-04-25T16:27:01.589509+0000","last_clean":"2024-04-25T16:27:01.589509+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:01.589509+0000","last_undegraded":"2024-04-25T16:27:01.589509+0000","last_fullsized":"2024-04-25T16:27:01.589509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584625+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:01.584625+0000","last_peered":"2024-04-25T16:27:01.584625+0000","last_clean":"2024-04-25T16:27:01.584625+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:01.584625+0000","last_undegraded":"2024-04-25T16:27:01.584625+0000","last_fullsized":"2024-04-25T16:27:01.584625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900487+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:02.900487+0000","last_peered":"2024-04-25T16:27:02.900487+0000","last_clean":"2024-04-25T16:27:02.900487+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:02.900487+0000","last_undegraded":"2024-04-25T16:27:02.900487+0000","last_fullsized":"2024-04-25T16:27:02.900487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161965+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:02.161965+0000","last_peered":"2024-04-25T16:27:02.161965+0000","last_clean":"2024-04-25T16:27:02.161965+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:02.161965+0000","last_undegraded":"2024-04-25T16:27:02.161965+0000","last_fullsized":"2024-04-25T16:27:02.161965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584525+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:01.584525+0000","last_peered":"2024-04-25T16:27:01.584525+0000","last_clean":"2024-04-25T16:27:01.584525+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:01.584525+0000","last_undegraded":"2024-04-25T16:27:01.584525+0000","last_fullsized":"2024-04-25T16:27:01.584525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902220+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:02.902220+0000","last_peered":"2024-04-25T16:27:02.902220+0000","last_clean":"2024-04-25T16:27:02.902220+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:02.902220+0000","last_undegraded":"2024-04-25T16:27:02.902220+0000","last_fullsized":"2024-04-25T16:27:02.902220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162131+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:02.162131+0000","last_peered":"2024-04-25T16:27:02.162131+0000","last_clean":"2024-04-25T16:27:02.162131+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:02.162131+0000","last_undegraded":"2024-04-25T16:27:02.162131+0000","last_fullsized":"2024-04-25T16:27:02.162131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900492+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:02.900492+0000","last_peered":"2024-04-25T16:27:02.900492+0000","last_clean":"2024-04-25T16:27:02.900492+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:02.900492+0000","last_undegraded":"2024-04-25T16:27:02.900492+0000","last_fullsized":"2024-04-25T16:27:02.900492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:07.253703+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:07.253703+0000","last_peered":"2024-04-25T16:27:07.253703+0000","last_clean":"2024-04-25T16:27:07.253703+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:07.253703+0000","last_undegraded":"2024-04-25T16:27:07.253703+0000","last_fullsized":"2024-04-25T16:27:07.253703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900859+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:02.900859+0000","last_peered":"2024-04-25T16:27:02.900859+0000","last_clean":"2024-04-25T16:27:02.900859+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:02.900859+0000","last_undegraded":"2024-04-25T16:27:02.900859+0000","last_fullsized":"2024-04-25T16:27:02.900859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590243+0000","last_change":"2024-04-25T16:26:56.570117+0000","last_active":"2024-04-25T16:27:01.590243+0000","last_peered":"2024-04-25T16:27:01.590243+0000","last_clean":"2024-04-25T16:27:01.590243+0000","last_became_active":"2024-04-25T16:26:56.569817+0000","last_became_peered":"2024-04-25T16:26:56.569817+0000","last_unstale":"2024-04-25T16:27:01.590243+0000","last_undegraded":"2024-04-25T16:27:01.590243+0000","last_fullsized":"2024-04-25T16:27:01.590243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:12.099635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900579+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:02.900579+0000","last_peered":"2024-04-25T16:27:02.900579+0000","last_clean":"2024-04-25T16:27:02.900579+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:02.900579+0000","last_undegraded":"2024-04-25T16:27:02.900579+0000","last_fullsized":"2024-04-25T16:27:02.900579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589933+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:01.589933+0000","last_peered":"2024-04-25T16:27:01.589933+0000","last_clean":"2024-04-25T16:27:01.589933+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:01.589933+0000","last_undegraded":"2024-04-25T16:27:01.589933+0000","last_fullsized":"2024-04-25T16:27:01.589933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901424+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:02.901424+0000","last_peered":"2024-04-25T16:27:02.901424+0000","last_clean":"2024-04-25T16:27:02.901424+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:02.901424+0000","last_undegraded":"2024-04-25T16:27:02.901424+0000","last_fullsized":"2024-04-25T16:27:02.901424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162032+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:02.162032+0000","last_peered":"2024-04-25T16:27:02.162032+0000","last_clean":"2024-04-25T16:27:02.162032+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:02.162032+0000","last_undegraded":"2024-04-25T16:27:02.162032+0000","last_fullsized":"2024-04-25T16:27:02.162032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584281+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:01.584281+0000","last_peered":"2024-04-25T16:27:01.584281+0000","last_clean":"2024-04-25T16:27:01.584281+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:01.584281+0000","last_undegraded":"2024-04-25T16:27:01.584281+0000","last_fullsized":"2024-04-25T16:27:01.584281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161984+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:02.161984+0000","last_peered":"2024-04-25T16:27:02.161984+0000","last_clean":"2024-04-25T16:27:02.161984+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:02.161984+0000","last_undegraded":"2024-04-25T16:27:02.161984+0000","last_fullsized":"2024-04-25T16:27:02.161984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591106+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:01.591106+0000","last_peered":"2024-04-25T16:27:01.591106+0000","last_clean":"2024-04-25T16:27:01.591106+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:01.591106+0000","last_undegraded":"2024-04-25T16:27:01.591106+0000","last_fullsized":"2024-04-25T16:27:01.591106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584144+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:01.584144+0000","last_peered":"2024-04-25T16:27:01.584144+0000","last_clean":"2024-04-25T16:27:01.584144+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:01.584144+0000","last_undegraded":"2024-04-25T16:27:01.584144+0000","last_fullsized":"2024-04-25T16:27:01.584144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901211+0000","last_change":"2024-04-25T16:27:00.605248+0000","last_active":"2024-04-25T16:27:02.901211+0000","last_peered":"2024-04-25T16:27:02.901211+0000","last_clean":"2024-04-25T16:27:02.901211+0000","last_became_active":"2024-04-25T16:27:00.604746+0000","last_became_peered":"2024-04-25T16:27:00.604746+0000","last_unstale":"2024-04-25T16:27:02.901211+0000","last_undegraded":"2024-04-25T16:27:02.901211+0000","last_fullsized":"2024-04-25T16:27:02.901211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900313+0000","last_change":"2024-04-25T16:26:56.553599+0000","last_active":"2024-04-25T16:27:02.900313+0000","last_peered":"2024-04-25T16:27:02.900313+0000","last_clean":"2024-04-25T16:27:02.900313+0000","last_became_active":"2024-04-25T16:26:56.553304+0000","last_became_peered":"2024-04-25T16:26:56.553304+0000","last_unstale":"2024-04-25T16:27:02.900313+0000","last_undegraded":"2024-04-25T16:27:02.900313+0000","last_fullsized":"2024-04-25T16:27:02.900313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161615+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:02.161615+0000","last_peered":"2024-04-25T16:27:02.161615+0000","last_clean":"2024-04-25T16:27:02.161615+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:02.161615+0000","last_undegraded":"2024-04-25T16:27:02.161615+0000","last_fullsized":"2024-04-25T16:27:02.161615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591033+0000","last_change":"2024-04-25T16:27:00.587907+0000","last_active":"2024-04-25T16:27:01.591033+0000","last_peered":"2024-04-25T16:27:01.591033+0000","last_clean":"2024-04-25T16:27:01.591033+0000","last_became_active":"2024-04-25T16:27:00.587281+0000","last_became_peered":"2024-04-25T16:27:00.587281+0000","last_unstale":"2024-04-25T16:27:01.591033+0000","last_undegraded":"2024-04-25T16:27:01.591033+0000","last_fullsized":"2024-04-25T16:27:01.591033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584123+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:01.584123+0000","last_peered":"2024-04-25T16:27:01.584123+0000","last_clean":"2024-04-25T16:27:01.584123+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:01.584123+0000","last_undegraded":"2024-04-25T16:27:01.584123+0000","last_fullsized":"2024-04-25T16:27:01.584123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584701+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:01.584701+0000","last_peered":"2024-04-25T16:27:01.584701+0000","last_clean":"2024-04-25T16:27:01.584701+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:01.584701+0000","last_undegraded":"2024-04-25T16:27:01.584701+0000","last_fullsized":"2024-04-25T16:27:01.584701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.591433+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:01.591433+0000","last_peered":"2024-04-25T16:27:01.591433+0000","last_clean":"2024-04-25T16:27:01.591433+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:01.591433+0000","last_undegraded":"2024-04-25T16:27:01.591433+0000","last_fullsized":"2024-04-25T16:27:01.591433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902190+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:02.902190+0000","last_peered":"2024-04-25T16:27:02.902190+0000","last_clean":"2024-04-25T16:27:02.902190+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:02.902190+0000","last_undegraded":"2024-04-25T16:27:02.902190+0000","last_fullsized":"2024-04-25T16:27:02.902190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900327+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:02.900327+0000","last_peered":"2024-04-25T16:27:02.900327+0000","last_clean":"2024-04-25T16:27:02.900327+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:02.900327+0000","last_undegraded":"2024-04-25T16:27:02.900327+0000","last_fullsized":"2024-04-25T16:27:02.900327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584664+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:01.584664+0000","last_peered":"2024-04-25T16:27:01.584664+0000","last_clean":"2024-04-25T16:27:01.584664+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:01.584664+0000","last_undegraded":"2024-04-25T16:27:01.584664+0000","last_fullsized":"2024-04-25T16:27:01.584664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589742+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:01.589742+0000","last_peered":"2024-04-25T16:27:01.589742+0000","last_clean":"2024-04-25T16:27:01.589742+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:01.589742+0000","last_undegraded":"2024-04-25T16:27:01.589742+0000","last_fullsized":"2024-04-25T16:27:01.589742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584547+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:01.584547+0000","last_peered":"2024-04-25T16:27:01.584547+0000","last_clean":"2024-04-25T16:27:01.584547+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:01.584547+0000","last_undegraded":"2024-04-25T16:27:01.584547+0000","last_fullsized":"2024-04-25T16:27:01.584547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589145+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:01.589145+0000","last_peered":"2024-04-25T16:27:01.589145+0000","last_clean":"2024-04-25T16:27:01.589145+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:01.589145+0000","last_undegraded":"2024-04-25T16:27:01.589145+0000","last_fullsized":"2024-04-25T16:27:01.589145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.899980+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:02.899980+0000","last_peered":"2024-04-25T16:27:02.899980+0000","last_clean":"2024-04-25T16:27:02.899980+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:02.899980+0000","last_undegraded":"2024-04-25T16:27:02.899980+0000","last_fullsized":"2024-04-25T16:27:02.899980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161855+0000","last_change":"2024-04-25T16:26:56.558625+0000","last_active":"2024-04-25T16:27:02.161855+0000","last_peered":"2024-04-25T16:27:02.161855+0000","last_clean":"2024-04-25T16:27:02.161855+0000","last_became_active":"2024-04-25T16:26:56.558322+0000","last_became_peered":"2024-04-25T16:26:56.558322+0000","last_unstale":"2024-04-25T16:27:02.161855+0000","last_undegraded":"2024-04-25T16:27:02.161855+0000","last_fullsized":"2024-04-25T16:27:02.161855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902073+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:02.902073+0000","last_peered":"2024-04-25T16:27:02.902073+0000","last_clean":"2024-04-25T16:27:02.902073+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:02.902073+0000","last_undegraded":"2024-04-25T16:27:02.902073+0000","last_fullsized":"2024-04-25T16:27:02.902073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584798+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:01.584798+0000","last_peered":"2024-04-25T16:27:01.584798+0000","last_clean":"2024-04-25T16:27:01.584798+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:01.584798+0000","last_undegraded":"2024-04-25T16:27:01.584798+0000","last_fullsized":"2024-04-25T16:27:01.584798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590378+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:01.590378+0000","last_peered":"2024-04-25T16:27:01.590378+0000","last_clean":"2024-04-25T16:27:01.590378+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:01.590378+0000","last_undegraded":"2024-04-25T16:27:01.590378+0000","last_fullsized":"2024-04-25T16:27:01.590378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584670+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:01.584670+0000","last_peered":"2024-04-25T16:27:01.584670+0000","last_clean":"2024-04-25T16:27:01.584670+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:01.584670+0000","last_undegraded":"2024-04-25T16:27:01.584670+0000","last_fullsized":"2024-04-25T16:27:01.584670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161594+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:02.161594+0000","last_peered":"2024-04-25T16:27:02.161594+0000","last_clean":"2024-04-25T16:27:02.161594+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:02.161594+0000","last_undegraded":"2024-04-25T16:27:02.161594+0000","last_fullsized":"2024-04-25T16:27:02.161594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584586+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:01.584586+0000","last_peered":"2024-04-25T16:27:01.584586+0000","last_clean":"2024-04-25T16:27:01.584586+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:01.584586+0000","last_undegraded":"2024-04-25T16:27:01.584586+0000","last_fullsized":"2024-04-25T16:27:01.584586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161810+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:02.161810+0000","last_peered":"2024-04-25T16:27:02.161810+0000","last_clean":"2024-04-25T16:27:02.161810+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:02.161810+0000","last_undegraded":"2024-04-25T16:27:02.161810+0000","last_fullsized":"2024-04-25T16:27:02.161810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901772+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:02.901772+0000","last_peered":"2024-04-25T16:27:02.901772+0000","last_clean":"2024-04-25T16:27:02.901772+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:02.901772+0000","last_undegraded":"2024-04-25T16:27:02.901772+0000","last_fullsized":"2024-04-25T16:27:02.901772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.589540+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:01.589540+0000","last_peered":"2024-04-25T16:27:01.589540+0000","last_clean":"2024-04-25T16:27:01.589540+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:01.589540+0000","last_undegraded":"2024-04-25T16:27:01.589540+0000","last_fullsized":"2024-04-25T16:27:01.589540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900759+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:02.900759+0000","last_peered":"2024-04-25T16:27:02.900759+0000","last_clean":"2024-04-25T16:27:02.900759+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:02.900759+0000","last_undegraded":"2024-04-25T16:27:02.900759+0000","last_fullsized":"2024-04-25T16:27:02.900759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901860+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:02.901860+0000","last_peered":"2024-04-25T16:27:02.901860+0000","last_clean":"2024-04-25T16:27:02.901860+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:02.901860+0000","last_undegraded":"2024-04-25T16:27:02.901860+0000","last_fullsized":"2024-04-25T16:27:02.901860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.162241+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:02.162241+0000","last_peered":"2024-04-25T16:27:02.162241+0000","last_clean":"2024-04-25T16:27:02.162241+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:02.162241+0000","last_undegraded":"2024-04-25T16:27:02.162241+0000","last_fullsized":"2024-04-25T16:27:02.162241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161412+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:02.161412+0000","last_peered":"2024-04-25T16:27:02.161412+0000","last_clean":"2024-04-25T16:27:02.161412+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:02.161412+0000","last_undegraded":"2024-04-25T16:27:02.161412+0000","last_fullsized":"2024-04-25T16:27:02.161412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900423+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:02.900423+0000","last_peered":"2024-04-25T16:27:02.900423+0000","last_clean":"2024-04-25T16:27:02.900423+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:02.900423+0000","last_undegraded":"2024-04-25T16:27:02.900423+0000","last_fullsized":"2024-04-25T16:27:02.900423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.902119+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:02.902119+0000","last_peered":"2024-04-25T16:27:02.902119+0000","last_clean":"2024-04-25T16:27:02.902119+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:02.902119+0000","last_undegraded":"2024-04-25T16:27:02.902119+0000","last_fullsized":"2024-04-25T16:27:02.902119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584351+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:01.584351+0000","last_peered":"2024-04-25T16:27:01.584351+0000","last_clean":"2024-04-25T16:27:01.584351+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:01.584351+0000","last_undegraded":"2024-04-25T16:27:01.584351+0000","last_fullsized":"2024-04-25T16:27:01.584351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590124+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:01.590124+0000","last_peered":"2024-04-25T16:27:01.590124+0000","last_clean":"2024-04-25T16:27:01.590124+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:01.590124+0000","last_undegraded":"2024-04-25T16:27:01.590124+0000","last_fullsized":"2024-04-25T16:27:01.590124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590011+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:01.590011+0000","last_peered":"2024-04-25T16:27:01.590011+0000","last_clean":"2024-04-25T16:27:01.590011+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:01.590011+0000","last_undegraded":"2024-04-25T16:27:01.590011+0000","last_fullsized":"2024-04-25T16:27:01.590011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.590256+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:01.590256+0000","last_peered":"2024-04-25T16:27:01.590256+0000","last_clean":"2024-04-25T16:27:01.590256+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:01.590256+0000","last_undegraded":"2024-04-25T16:27:01.590256+0000","last_fullsized":"2024-04-25T16:27:01.590256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900847+0000","last_change":"2024-04-25T16:26:56.572739+0000","last_active":"2024-04-25T16:27:02.900847+0000","last_peered":"2024-04-25T16:27:02.900847+0000","last_clean":"2024-04-25T16:27:02.900847+0000","last_became_active":"2024-04-25T16:26:56.571251+0000","last_became_peered":"2024-04-25T16:26:56.571251+0000","last_unstale":"2024-04-25T16:27:02.900847+0000","last_undegraded":"2024-04-25T16:27:02.900847+0000","last_fullsized":"2024-04-25T16:27:02.900847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:01.584005+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:01.584005+0000","last_peered":"2024-04-25T16:27:01.584005+0000","last_clean":"2024-04-25T16:27:01.584005+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:01.584005+0000","last_undegraded":"2024-04-25T16:27:01.584005+0000","last_fullsized":"2024-04-25T16:27:01.584005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900188+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:02.900188+0000","last_peered":"2024-04-25T16:27:02.900188+0000","last_clean":"2024-04-25T16:27:02.900188+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:02.900188+0000","last_undegraded":"2024-04-25T16:27:02.900188+0000","last_fullsized":"2024-04-25T16:27:02.900188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161603+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:02.161603+0000","last_peered":"2024-04-25T16:27:02.161603+0000","last_clean":"2024-04-25T16:27:02.161603+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:02.161603+0000","last_undegraded":"2024-04-25T16:27:02.161603+0000","last_fullsized":"2024-04-25T16:27:02.161603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900383+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:02.900383+0000","last_peered":"2024-04-25T16:27:02.900383+0000","last_clean":"2024-04-25T16:27:02.900383+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:02.900383+0000","last_undegraded":"2024-04-25T16:27:02.900383+0000","last_fullsized":"2024-04-25T16:27:02.900383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.161799+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:02.161799+0000","last_peered":"2024-04-25T16:27:02.161799+0000","last_clean":"2024-04-25T16:27:02.161799+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:02.161799+0000","last_undegraded":"2024-04-25T16:27:02.161799+0000","last_fullsized":"2024-04-25T16:27:02.161799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900832+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:02.900832+0000","last_peered":"2024-04-25T16:27:02.900832+0000","last_clean":"2024-04-25T16:27:02.900832+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:02.900832+0000","last_undegraded":"2024-04-25T16:27:02.900832+0000","last_fullsized":"2024-04-25T16:27:02.900832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.901018+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:02.901018+0000","last_peered":"2024-04-25T16:27:02.901018+0000","last_clean":"2024-04-25T16:27:02.901018+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:02.901018+0000","last_undegraded":"2024-04-25T16:27:02.901018+0000","last_fullsized":"2024-04-25T16:27:02.901018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T16:27:02.900715+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:02.900715+0000","last_peered":"2024-04-25T16:27:02.900715+0000","last_clean":"2024-04-25T16:27:02.900715+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:02.900715+0000","last_undegraded":"2024-04-25T16:27:02.900715+0000","last_fullsized":"2024-04-25T16:27:02.900715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":103933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":274432,"data_stored":107649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":103807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":103967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":694813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:27.693 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T16:27:27.693 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T16:27:27.693 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T16:27:27.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T16:27:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.835+0000 7ff0f50d8640 1 Processor -- start 2024-04-25T16:27:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.835+0000 7ff0f50d8640 1 -- start start 2024-04-25T16:27:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.836+0000 7ff0f50d8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff0f0162a10 0x7ff0f00afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.836+0000 7ff0f50d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.836+0000 7ff0f50d8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0f00b0530 con 0x7ff0f00aedd0 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.836+0000 7ff0f50d8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f006ded0 con 0x7ff0f0162a10 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.836+0000 7ff0f50d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f0162e20 con 0x7ff0f006d580 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0ef59e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff0f00aedd0 0x7ff0f00af1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56328/0 (socket says 172.21.15.94:56328) 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0ef59e640 1 -- 172.21.15.94:0/2147159052 learned_addr learned my addr 172.21.15.94:0/2147159052 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135821598 0 0) 0x7ff0e8001110 con 0x7ff0f00aedd0 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0d8003520 con 0x7ff0f00aedd0 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/2147159052 >> v2:172.21.15.138:3300/0 conn(0x7ff0f0162a10 0x7ff0f00afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0ee59c640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3440132396 0 0) 0x7ff0d8003520 con 0x7ff0f00aedd0 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 msgr2=0x7ff0f006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0f006d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 >> v2:172.21.15.138:3300/0 conn(0x7ff0f0162a10 msgr2=0x7ff0f00afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 --2- 172.21.15.94:0/2147159052 >> v2:172.21.15.138:3300/0 conn(0x7ff0f0162a10 0x7ff0f00afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f00b07b0 con 0x7ff0f00aedd0 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7ff0e8001860 con 0x7ff0f00aedd0 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 >> v1:172.21.15.94:6789/0 conn(0x7ff0f00aedd0 legacy=0x7ff0f00af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0d80039b0 0x7ff0d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0d8004800 0x7ff0d8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0d8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f006ded0 con 0x7ff0d80039b0 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0162e20 con 0x7ff0d8004800 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d800baa0 con 0x7ff0f006d580 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0d80039b0 0x7ff0d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0ee59c640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0d8004800 0x7ff0d8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0ef59e640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0d8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.837+0000 7ff0eed9d640 1 -- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 msgr2=0x7ff0d8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0d8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0eed9d640 1 -- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0d8004800 msgr2=0x7ff0d8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0d8004800 0x7ff0d8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0eed9d640 1 -- 172.21.15.94:0/2147159052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d8003520 con 0x7ff0d80039b0 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0ef59e640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0f006d580 0x7ff0d8005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0d80039b0 0x7ff0d8003e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0073e50 tx=0x7ff0d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637f41fee193948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0009c10 con 0x7ff0d80039b0 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e0036070 con 0x7ff0d80039b0 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.838+0000 7ff0edd9b640 1 -- 172.21.15.94:0/2147159052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0031810 con 0x7ff0d80039b0 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0f50d8640 1 -- 172.21.15.94:0/2147159052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0d80039b0 msgr2=0x7ff0d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/2147159052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0d80039b0 0x7ff0d8003e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0073e50 tx=0x7ff0d8005830 comp rx=0 tx=0).stop 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0ef59e640 1 -- 172.21.15.94:0/2147159052 reap_dead start 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0f50d8640 1 -- 172.21.15.94:0/2147159052 shutdown_connections 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0f50d8640 1 -- 172.21.15.94:0/2147159052 >> 172.21.15.94:0/2147159052 conn(0x7ff0f00d23b0 msgr2=0x7ff0f0073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0f50d8640 1 -- 172.21.15.94:0/2147159052 shutdown_connections 2024-04-25T16:27:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.839+0000 7ff0f50d8640 1 -- 172.21.15.94:0/2147159052 wait complete. 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.841+0000 7ff0f50d8640 1 Processor -- start 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.841+0000 7ff0f50d8640 1 -- start start 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.842+0000 7ff0f50d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0d80039b0 0x7ff0f015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.842+0000 7ff0f50d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0f00cbcf0 0x7ff0f00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.842+0000 7ff0f50d8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 0x7ff0f00d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.842+0000 7ff0f50d8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f015f250 con 0x7ff0f009b9b0 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.842+0000 7ff0f50d8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f015ef70 con 0x7ff0f00cbcf0 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.842+0000 7ff0f50d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f015e980 con 0x7ff0d80039b0 2024-04-25T16:27:27.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 0x7ff0f00d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 0x7ff0f00d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56274/0 (socket says 172.21.15.94:56274) 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 -- 172.21.15.94:0/1577088123 learned_addr learned my addr 172.21.15.94:0/1577088123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ee59c640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0f00cbcf0 0x7ff0f00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0d80039b0 0x7ff0f015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 -- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0d80039b0 msgr2=0x7ff0f015d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0d80039b0 0x7ff0f015d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 -- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0f00cbcf0 msgr2=0x7ff0f00ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0f00cbcf0 0x7ff0f00ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 -- 172.21.15.94:0/1577088123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f015dec0 con 0x7ff0f009b9b0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ee59c640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0f00cbcf0 0x7ff0f00ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0d80039b0 0x7ff0f015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0ef59e640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 0x7ff0f00d0200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00b13b0 tx=0x7ff0e8007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8010510 con 0x7ff0f009b9b0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e80106b0 con 0x7ff0f009b9b0 2024-04-25T16:27:27.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f0143ec0 con 0x7ff0f009b9b0 2024-04-25T16:27:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8010a60 con 0x7ff0f009b9b0 2024-04-25T16:27:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.843+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f00cd980 con 0x7ff0f009b9b0 2024-04-25T16:27:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.844+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f00cd6f0 con 0x7ff0f009b9b0 2024-04-25T16:27:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.845+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff0e801b020 con 0x7ff0f009b9b0 2024-04-25T16:27:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.845+0000 7ff0d77fe640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff0dc0744e0 0x7ff0dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:27.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.845+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff0dc0744e0 0x7ff0dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.845+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff0e8094a20 con 0x7ff0f009b9b0 2024-04-25T16:27:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.846+0000 7ff0eed9d640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff0dc0744e0 0x7ff0dc0769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0035000 tx=0x7ff0e002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:27.850+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0f00b0530 con 0x7ff0f009b9b0 2024-04-25T16:27:28.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.064+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff0f009c580 con 0x7ff0f009b9b0 2024-04-25T16:27:28.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.065+0000 7ff0d77fe640 1 -- 172.21.15.94:0/1577088123 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff0e8065e20 con 0x7ff0f009b9b0 2024-04-25T16:27:28.065 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:28.066 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff0dc0744e0 msgr2=0x7ff0dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff0dc0744e0 0x7ff0dc0769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0035000 tx=0x7ff0e002f000 comp rx=0 tx=0).stop 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 msgr2=0x7ff0f00d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 0x7ff0f00d0200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00b13b0 tx=0x7ff0e8007420 comp rx=0 tx=0).stop 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 shutdown_connections 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff0dc0744e0 0x7ff0dc0769f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0f009b9b0 0x7ff0f00d0200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0f00cbcf0 0x7ff0f00ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 --2- 172.21.15.94:0/1577088123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0d80039b0 0x7ff0f015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 >> 172.21.15.94:0/1577088123 conn(0x7ff0f00d23b0 msgr2=0x7ff0f006f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 shutdown_connections 2024-04-25T16:27:28.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.067+0000 7ff0f50d8640 1 -- 172.21.15.94:0/1577088123 wait complete. 2024-04-25T16:27:28.079 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T16:27:28.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:28.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.185+0000 7fa5631ba640 1 Processor -- start 2024-04-25T16:27:28.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.185+0000 7fa5631ba640 1 -- start start 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa5631ba640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa55c0d4bb0 0x7fa55c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa5631ba640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55c06c9c0 con 0x7fa55c0d1010 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa5631ba640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55c06cca0 con 0x7fa55c08e780 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa5631ba640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c06dfa0 con 0x7fa55c0d4bb0 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa561757640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa55c0d1010 0x7fa55c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56344/0 (socket says 172.21.15.94:56344) 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa561757640 1 -- 172.21.15.94:0/3624595367 learned_addr learned my addr 172.21.15.94:0/3624595367 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55bfff640 1 --2- 172.21.15.94:0/3624595367 >> v2:172.21.15.138:3300/0 conn(0x7fa55c0d4bb0 0x7fa55c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55b7fe640 1 -- 172.21.15.94:0/3624595367 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2441976328 0 0) 0x7fa55c06c9c0 con 0x7fa55c0d1010 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55b7fe640 1 -- 172.21.15.94:0/3624595367 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa544003520 con 0x7fa55c0d1010 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55bfff640 1 -- 172.21.15.94:0/3624595367 >> v1:172.21.15.138:6789/0 conn(0x7fa55c08e780 legacy=0x7fa55c0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55bfff640 1 -- 172.21.15.94:0/3624595367 >> v1:172.21.15.94:6789/0 conn(0x7fa55c0d1010 legacy=0x7fa55c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55b7fe640 1 -- 172.21.15.94:0/3624595367 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3819447766 0 0) 0x7fa544003520 con 0x7fa55c0d1010 2024-04-25T16:27:28.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.187+0000 7fa55bfff640 1 -- 172.21.15.94:0/3624595367 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c06cf80 con 0x7fa55c0d4bb0 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.188+0000 7fa55bfff640 1 --2- 172.21.15.94:0/3624595367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c06da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5500097a0 tx=0x7fa55002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37bda8742a44d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.188+0000 7fa55b7fe640 1 -- 172.21.15.94:0/3624595367 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550037070 con 0x7fa55c0d4bb0 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.188+0000 7fa55b7fe640 1 -- 172.21.15.94:0/3624595367 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55002abc0 con 0x7fa55c0d4bb0 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.188+0000 7fa55b7fe640 1 -- 172.21.15.94:0/3624595367 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5540019f0 con 0x7fa55c0d4bb0 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 -- 172.21.15.94:0/3624595367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 msgr2=0x7fa55c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 --2- 172.21.15.94:0/3624595367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c06da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5500097a0 tx=0x7fa55002a130 comp rx=0 tx=0).stop 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 -- 172.21.15.94:0/3624595367 shutdown_connections 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 --2- 172.21.15.94:0/3624595367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c06da60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 -- 172.21.15.94:0/3624595367 >> 172.21.15.94:0/3624595367 conn(0x7fa55c0b7cc0 msgr2=0x7fa55c0b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:28.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 -- 172.21.15.94:0/3624595367 shutdown_connections 2024-04-25T16:27:28.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.189+0000 7fa5631ba640 1 -- 172.21.15.94:0/3624595367 wait complete. 2024-04-25T16:27:28.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.191+0000 7fa5631ba640 1 Processor -- start 2024-04-25T16:27:28.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.191+0000 7fa5631ba640 1 -- start start 2024-04-25T16:27:28.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa5631ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 0x7fa55c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa5631ba640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa55c0d1010 0x7fa55c0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa5631ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa5631ba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c06c570 con 0x7fa55c08e780 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa5631ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c06d540 con 0x7fa55c0d4bb0 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa5631ba640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c06c6f0 con 0x7fa55c0d1010 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa560f56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 0x7fa55c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa560f56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 0x7fa55c150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56276/0 (socket says 172.21.15.94:56276) 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa560f56640 1 -- 172.21.15.94:0/1807845677 learned_addr learned my addr 172.21.15.94:0/1807845677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa561757640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.192+0000 7fa55bfff640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa55c0d1010 0x7fa55c0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa560f56640 1 -- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa55c0d1010 msgr2=0x7fa55c0b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa560f56640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa55c0d1010 0x7fa55c0b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa560f56640 1 -- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 msgr2=0x7fa55c0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa560f56640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa560f56640 1 -- 172.21.15.94:0/1807845677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c0b3cc0 con 0x7fa55c08e780 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa561757640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa55bfff640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa55c0d1010 0x7fa55c0b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:28.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa560f56640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 0x7fa55c150bd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c430 tx=0x7fa54c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c017070 con 0x7fa55c08e780 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c0052d0 con 0x7fa55c08e780 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c003830 con 0x7fa55c08e780 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c0b3fa0 con 0x7fa55c08e780 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.193+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c0b4560 con 0x7fa55c08e780 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.194+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c06a180 con 0x7fa55c08e780 2024-04-25T16:27:28.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.195+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa54c01a070 con 0x7fa55c08e780 2024-04-25T16:27:28.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.195+0000 7fa5597fa640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa5480744b0 0x7fa5480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.195+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa54c094900 con 0x7fa55c08e780 2024-04-25T16:27:28.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.195+0000 7fa55bfff640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa5480744b0 0x7fa5480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.196+0000 7fa55bfff640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa5480744b0 0x7fa5480769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa550034000 tx=0x7fa55002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.200+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa54c05ed50 con 0x7fa55c08e780 2024-04-25T16:27:28.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.377+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa55c0b7710 con 0x7fa55c08e780 2024-04-25T16:27:28.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.378+0000 7fa5597fa640 1 -- 172.21.15.94:0/1807845677 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fa54c05e650 con 0x7fa55c08e780 2024-04-25T16:27:28.379 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:28.379 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:28.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa5480744b0 msgr2=0x7fa5480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa5480744b0 0x7fa5480769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa550034000 tx=0x7fa55002e000 comp rx=0 tx=0).stop 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 msgr2=0x7fa55c150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 0x7fa55c150bd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c430 tx=0x7fa54c00c8f0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 shutdown_connections 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa5480744b0 0x7fa5480769c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa55c0d4bb0 0x7fa55c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa55c0d1010 0x7fa55c0b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 --2- 172.21.15.94:0/1807845677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa55c08e780 0x7fa55c150bd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 >> 172.21.15.94:0/1807845677 conn(0x7fa55c0b7cc0 msgr2=0x7fa55c0b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 shutdown_connections 2024-04-25T16:27:28.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.381+0000 7fa5631ba640 1 -- 172.21.15.94:0/1807845677 wait complete. 2024-04-25T16:27:28.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:28.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:28.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.503+0000 7f1347070640 1 Processor -- start 2024-04-25T16:27:28.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.503+0000 7f1347070640 1 -- start start 2024-04-25T16:27:28.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f1347070640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1340168180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f1347070640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1340161920 con 0x7f134015a9f0 2024-04-25T16:27:28.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f1347070640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134015ae00 con 0x7f13400d0dd0 2024-04-25T16:27:28.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f1347070640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f134006d580 con 0x7f1340165d60 2024-04-25T16:27:28.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f1344e0c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f134015a9f0 0x7f13400707f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56360/0 (socket says 172.21.15.94:56360) 2024-04-25T16:27:28.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f1344e0c640 1 -- 172.21.15.94:0/224955530 learned_addr learned my addr 172.21.15.94:0/224955530 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f134560d640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1340168180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604134386 0 0) 0x7f1340161920 con 0x7f134015a9f0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1328003520 con 0x7f134015a9f0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209247105 0 0) 0x7f134015ae00 con 0x7f13400d0dd0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1340161920 con 0x7f13400d0dd0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3368481057 0 0) 0x7f1328003520 con 0x7f134015a9f0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 msgr2=0x7f1340168180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1340168180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 >> v1:172.21.15.138:6789/0 conn(0x7f13400d0dd0 legacy=0x7f13400d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1340070f00 con 0x7f134015a9f0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f1330001670 con 0x7f134015a9f0 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f134560d640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1340168180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 >> v1:172.21.15.94:6789/0 conn(0x7f134015a9f0 legacy=0x7f13400707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1328003b70 0x7f1328003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1328004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328004d40 0x7f1328005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f132800ba70 con 0x7f1328003b70 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f132800bc20 con 0x7f1328004d40 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.505+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f132800be20 con 0x7f1340165d60 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f1344e0c640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1328003b70 0x7f1328003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f133ffff640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1328004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f134560d640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328004d40 0x7f1328005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f1344e0c640 1 -- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 msgr2=0x7f1328004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f1344e0c640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1340165d60 0x7f1328004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f1344e0c640 1 -- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328004d40 msgr2=0x7f1328005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f1344e0c640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328004d40 0x7f1328005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.506+0000 7f1344e0c640 1 -- 172.21.15.94:0/224955530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1328003520 con 0x7f1328003b70 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1344e0c640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1328003b70 0x7f1328003fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f133000c190 tx=0x7f1328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18feb0c68e15082b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330007cc0 con 0x7f1328003b70 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1330005ad0 con 0x7f1328003b70 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f133f7fe640 1 -- 172.21.15.94:0/224955530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330007070 con 0x7f1328003b70 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1347070640 1 -- 172.21.15.94:0/224955530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1328003b70 msgr2=0x7f1328003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1347070640 1 --2- 172.21.15.94:0/224955530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1328003b70 0x7f1328003fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f133000c190 tx=0x7f1328005830 comp rx=0 tx=0).stop 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f134560d640 1 -- 172.21.15.94:0/224955530 reap_dead start 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1347070640 1 -- 172.21.15.94:0/224955530 shutdown_connections 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1347070640 1 -- 172.21.15.94:0/224955530 >> 172.21.15.94:0/224955530 conn(0x7f13400b4270 msgr2=0x7f13400ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1347070640 1 -- 172.21.15.94:0/224955530 shutdown_connections 2024-04-25T16:27:28.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.507+0000 7f1347070640 1 -- 172.21.15.94:0/224955530 wait complete. 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.509+0000 7f1347070640 1 Processor -- start 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.509+0000 7f1347070640 1 -- start start 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328003b70 0x7f134015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13400b6320 0x7f1340152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 0x7f13400b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1340160e40 con 0x7f13400b22a0 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13400711b0 con 0x7f1328003b70 2024-04-25T16:27:28.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1340161120 con 0x7f13400b6320 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 0x7f13400b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 0x7f13400b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56288/0 (socket says 172.21.15.94:56288) 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 -- 172.21.15.94:0/414799100 learned_addr learned my addr 172.21.15.94:0/414799100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f133ffff640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13400b6320 0x7f1340152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1344e0c640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328003b70 0x7f134015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 -- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13400b6320 msgr2=0x7f1340152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13400b6320 0x7f1340152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 -- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328003b70 msgr2=0x7f134015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328003b70 0x7f134015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 -- 172.21.15.94:0/414799100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13400b2bf0 con 0x7f13400b22a0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f134560d640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 0x7f13400b26b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f134006e130 tx=0x7f1338010580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133800a2e0 con 0x7f13400b22a0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338010aa0 con 0x7f13400b22a0 2024-04-25T16:27:28.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1340153250 con 0x7f13400b22a0 2024-04-25T16:27:28.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338010e70 con 0x7f13400b22a0 2024-04-25T16:27:28.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.511+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134015f5d0 con 0x7f13400b22a0 2024-04-25T16:27:28.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.513+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134006a180 con 0x7f13400b22a0 2024-04-25T16:27:28.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.513+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f133801f020 con 0x7f13400b22a0 2024-04-25T16:27:28.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.514+0000 7f133d7fa640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f132c0744e0 0x7f132c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.514+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1338096260 con 0x7f13400b22a0 2024-04-25T16:27:28.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.514+0000 7f1344e0c640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f132c0744e0 0x7f132c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.514+0000 7f1344e0c640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f132c0744e0 0x7f132c0769f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1330032000 tx=0x7f1330009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.518+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1338026020 con 0x7f13400b22a0 2024-04-25T16:27:28.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.697+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13400ce700 con 0x7f13400b22a0 2024-04-25T16:27:28.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.697+0000 7f133d7fa640 1 -- 172.21.15.94:0/414799100 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f13400b2bf0 con 0x7f13400b22a0 2024-04-25T16:27:28.698 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f132c0744e0 msgr2=0x7f132c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f132c0744e0 0x7f132c0769f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1330032000 tx=0x7f1330009030 comp rx=0 tx=0).stop 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 msgr2=0x7f13400b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 0x7f13400b26b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f134006e130 tx=0x7f1338010580 comp rx=0 tx=0).stop 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 shutdown_connections 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f132c0744e0 0x7f132c0769f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13400b22a0 0x7f13400b26b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13400b6320 0x7f1340152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 --2- 172.21.15.94:0/414799100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1328003b70 0x7f134015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 >> 172.21.15.94:0/414799100 conn(0x7f13400b4270 msgr2=0x7f1340062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 shutdown_connections 2024-04-25T16:27:28.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.699+0000 7f1347070640 1 -- 172.21.15.94:0/414799100 wait complete. 2024-04-25T16:27:28.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.820+0000 7f4b25ce7640 1 Processor -- start 2024-04-25T16:27:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.820+0000 7f4b25ce7640 1 -- start start 2024-04-25T16:27:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b25ce7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4b200b9780 0x7f4b2009c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b25ce7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b2006ea50 0x7f4b200ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b25ce7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b200d39e0 con 0x7f4b2009c7f0 2024-04-25T16:27:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b25ce7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4b200d2dd0 con 0x7f4b200b9780 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b25ce7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b200d30b0 con 0x7f4b2006ea50 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4b200b9780 0x7f4b2009c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4b200b9780 0x7f4b2009c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56300/0 (socket says 172.21.15.94:56300) 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 -- 172.21.15.94:0/2134885659 learned_addr learned my addr 172.21.15.94:0/2134885659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1ffff640 1 --2- 172.21.15.94:0/2134885659 >> v2:172.21.15.138:3300/0 conn(0x7f4b2006ea50 0x7f4b200ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 -- 172.21.15.94:0/2134885659 >> v2:172.21.15.138:3300/0 conn(0x7f4b2006ea50 msgr2=0x7f4b200ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 --2- 172.21.15.94:0/2134885659 >> v2:172.21.15.138:3300/0 conn(0x7f4b2006ea50 0x7f4b200ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 -- 172.21.15.94:0/2134885659 >> v1:172.21.15.138:6789/0 conn(0x7f4b2009c7f0 legacy=0x7f4b2006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1f7fe640 1 -- 172.21.15.94:0/2134885659 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200d3390 con 0x7f4b200b9780 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.821+0000 7f4b1ffff640 1 --2- 172.21.15.94:0/2134885659 >> v2:172.21.15.138:3300/0 conn(0x7f4b2006ea50 0x7f4b200ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.822+0000 7f4b1f7fe640 1 --2- 172.21.15.94:0/2134885659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b200b9780 0x7f4b2009c2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b200cb090 tx=0x7f4b1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d435783cb58c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.822+0000 7f4b1e7fc640 1 -- 172.21.15.94:0/2134885659 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10037070 con 0x7f4b200b9780 2024-04-25T16:27:28.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.822+0000 7f4b1e7fc640 1 -- 172.21.15.94:0/2134885659 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b10032440 con 0x7f4b200b9780 2024-04-25T16:27:28.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.822+0000 7f4b1e7fc640 1 -- 172.21.15.94:0/2134885659 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10032810 con 0x7f4b200b9780 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 -- 172.21.15.94:0/2134885659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b200b9780 msgr2=0x7f4b2009c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/2134885659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b200b9780 0x7f4b2009c2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b200cb090 tx=0x7f4b1002a270 comp rx=0 tx=0).stop 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 -- 172.21.15.94:0/2134885659 shutdown_connections 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/2134885659 >> v2:172.21.15.138:3300/0 conn(0x7f4b2006ea50 0x7f4b200ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/2134885659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b200b9780 0x7f4b2009c2b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 -- 172.21.15.94:0/2134885659 >> 172.21.15.94:0/2134885659 conn(0x7f4b200b5b20 msgr2=0x7f4b200b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 -- 172.21.15.94:0/2134885659 shutdown_connections 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.823+0000 7f4b25ce7640 1 -- 172.21.15.94:0/2134885659 wait complete. 2024-04-25T16:27:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.825+0000 7f4b25ce7640 1 Processor -- start 2024-04-25T16:27:28.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.825+0000 7f4b25ce7640 1 -- start start 2024-04-25T16:27:28.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b25ce7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b2006ea50 0x7f4b2015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b25ce7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 0x7f4b20150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b25ce7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b200b9780 0x7f4b20160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b25ce7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b50e0 con 0x7f4b2009c7f0 2024-04-25T16:27:28.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b25ce7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b4f60 con 0x7f4b200b9780 2024-04-25T16:27:28.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b25ce7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2009d040 con 0x7f4b2006ea50 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b1effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 0x7f4b20150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b1effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 0x7f4b20150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56306/0 (socket says 172.21.15.94:56306) 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.826+0000 7f4b1effd640 1 -- 172.21.15.94:0/3746886053 learned_addr learned my addr 172.21.15.94:0/3746886053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1ffff640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b200b9780 0x7f4b20160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1f7fe640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b2006ea50 0x7f4b2015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1effd640 1 -- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b2006ea50 msgr2=0x7f4b2015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1effd640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b2006ea50 0x7f4b2015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1effd640 1 -- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b200b9780 msgr2=0x7f4b20160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1effd640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b200b9780 0x7f4b20160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1effd640 1 -- 172.21.15.94:0/3746886053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2015de10 con 0x7f4b2009c7f0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1ffff640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b200b9780 0x7f4b20160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b1effd640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 0x7f4b20150390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c340 tx=0x7f4b1400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14017070 con 0x7f4b2009c7f0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b140052d0 con 0x7f4b2009c7f0 2024-04-25T16:27:28.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2015e0f0 con 0x7f4b2009c7f0 2024-04-25T16:27:28.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14003850 con 0x7f4b2009c7f0 2024-04-25T16:27:28.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.827+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2015e6b0 con 0x7f4b2009c7f0 2024-04-25T16:27:28.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.828+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2015db80 con 0x7f4b2009c7f0 2024-04-25T16:27:28.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.829+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b1401a070 con 0x7f4b2009c7f0 2024-04-25T16:27:28.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.829+0000 7f4b07fff640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b0c0744b0 0x7f4b0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:28.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.829+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4b14094900 con 0x7f4b2009c7f0 2024-04-25T16:27:28.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.829+0000 7f4b1f7fe640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b0c0744b0 0x7f4b0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:28.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.830+0000 7f4b1f7fe640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b0c0744b0 0x7f4b0c0769c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b10034000 tx=0x7f4b1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:28.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:28.834+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b1405ecd0 con 0x7f4b2009c7f0 2024-04-25T16:27:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.008+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b2006a180 con 0x7f4b2009c7f0 2024-04-25T16:27:29.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.009+0000 7f4b07fff640 1 -- 172.21.15.94:0/3746886053 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f4b1405e5d0 con 0x7f4b2009c7f0 2024-04-25T16:27:29.011 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b0c0744b0 msgr2=0x7f4b0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b0c0744b0 0x7f4b0c0769c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b10034000 tx=0x7f4b1002e000 comp rx=0 tx=0).stop 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 msgr2=0x7f4b20150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 0x7f4b20150390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c340 tx=0x7f4b1400c800 comp rx=0 tx=0).stop 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 shutdown_connections 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4b0c0744b0 0x7f4b0c0769c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b200b9780 0x7f4b20160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b2009c7f0 0x7f4b20150390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 --2- 172.21.15.94:0/3746886053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b2006ea50 0x7f4b2015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 >> 172.21.15.94:0/3746886053 conn(0x7f4b200b5b20 msgr2=0x7f4b201630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 shutdown_connections 2024-04-25T16:27:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.011+0000 7f4b25ce7640 1 -- 172.21.15.94:0/3746886053 wait complete. 2024-04-25T16:27:29.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:29.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.133+0000 7f700a9f3640 1 Processor -- start 2024-04-25T16:27:29.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.133+0000 7f700a9f3640 1 -- start start 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.134+0000 7f700a9f3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f70041650b0 0x7f70040b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.134+0000 7f700a9f3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f700406cdc0 con 0x7f700406c570 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.134+0000 7f700a9f3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f700406d0a0 con 0x7f70040d18a0 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.134+0000 7f700a9f3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f700406d380 con 0x7f70041650b0 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.134+0000 7f70037fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f700406c570 0x7f70040d1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56366/0 (socket says 172.21.15.94:56366) 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f70037fe640 1 -- 172.21.15.94:0/2721733250 learned_addr learned my addr 172.21.15.94:0/2721733250 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:29.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7003fff640 1 --2- 172.21.15.94:0/2721733250 >> v2:172.21.15.138:3300/0 conn(0x7f70041650b0 0x7f70040b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921378429 0 0) 0x7f700406cdc0 con 0x7f700406c570 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fec003520 con 0x7f700406c570 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3700722246 0 0) 0x7f6fec003520 con 0x7f700406c570 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 >> v2:172.21.15.138:3300/0 conn(0x7f70041650b0 msgr2=0x7f70040b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 --2- 172.21.15.94:0/2721733250 >> v2:172.21.15.138:3300/0 conn(0x7f70041650b0 0x7f70040b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 >> v1:172.21.15.138:6789/0 conn(0x7f70040d18a0 legacy=0x7f7004161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700406d660 con 0x7f700406c570 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f6ff8001690 con 0x7f700406c570 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 >> v1:172.21.15.94:6789/0 conn(0x7f700406c570 legacy=0x7f70040d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec0039b0 0x7f6fec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fec004800 0x7f6fec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fec005250 0x7f6fec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f700406d380 con 0x7f6fec005250 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec00c1e0 con 0x7f6fec0039b0 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6fec00c3e0 con 0x7f6fec004800 2024-04-25T16:27:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7008f90640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fec005250 0x7f6fec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f7003fff640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec0039b0 0x7f6fec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.135+0000 7f70037fe640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fec004800 0x7f6fec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 -- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fec004800 msgr2=0x7f6fec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fec004800 0x7f6fec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 -- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec0039b0 msgr2=0x7f6fec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec0039b0 0x7f6fec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 -- 172.21.15.94:0/2721733250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec003520 con 0x7f6fec005250 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 -- 172.21.15.94:0/2721733250 reap_dead start 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7008f90640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fec005250 0x7f6fec00b9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc001e20 tx=0x7f6fec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681aec445eb0caaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc001a70 con 0x7f6fec005250 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ffc02ae40 con 0x7f6fec005250 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.136+0000 7f7002ffd640 1 -- 172.21.15.94:0/2721733250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc003bb0 con 0x7f6fec005250 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 -- 172.21.15.94:0/2721733250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fec005250 msgr2=0x7f6fec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fec005250 0x7f6fec00b9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc001e20 tx=0x7f6fec005830 comp rx=0 tx=0).stop 2024-04-25T16:27:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 -- 172.21.15.94:0/2721733250 shutdown_connections 2024-04-25T16:27:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 --2- 172.21.15.94:0/2721733250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fec005250 0x7f6fec00b9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 -- 172.21.15.94:0/2721733250 >> 172.21.15.94:0/2721733250 conn(0x7f70040b7e90 msgr2=0x7f70040b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 -- 172.21.15.94:0/2721733250 shutdown_connections 2024-04-25T16:27:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.137+0000 7f700a9f3640 1 -- 172.21.15.94:0/2721733250 wait complete. 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.139+0000 7f700a9f3640 1 Processor -- start 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.139+0000 7f700a9f3640 1 -- start start 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.140+0000 7f700a9f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec005250 0x7f7004148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.140+0000 7f700a9f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 0x7f70041636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.140+0000 7f700a9f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7004163c00 0x7f70040baa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.140+0000 7f700a9f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f700406c980 con 0x7f70040cfd60 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.140+0000 7f700a9f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f700406de60 con 0x7f6fec005250 2024-04-25T16:27:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.140+0000 7f700a9f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f700406cb00 con 0x7f7004163c00 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 0x7f70041636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 0x7f70041636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56330/0 (socket says 172.21.15.94:56330) 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 -- 172.21.15.94:0/531872760 learned_addr learned my addr 172.21.15.94:0/531872760 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f7003fff640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec005250 0x7f7004148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f7008f90640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7004163c00 0x7f70040baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 -- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7004163c00 msgr2=0x7f70040baa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7004163c00 0x7f70040baa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 -- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec005250 msgr2=0x7f7004148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec005250 0x7f7004148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 -- 172.21.15.94:0/531872760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70041559a0 con 0x7f70040cfd60 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f7003fff640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec005250 0x7f7004148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f70037fe640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 0x7f70041636c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007340 tx=0x7f6ff8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800f940 con 0x7f70040cfd60 2024-04-25T16:27:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff8018070 con 0x7f70040cfd60 2024-04-25T16:27:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7004148570 con 0x7f70040cfd60 2024-04-25T16:27:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800fd60 con 0x7f70040cfd60 2024-04-25T16:27:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.141+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7004145590 con 0x7f70040cfd60 2024-04-25T16:27:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.142+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700406a180 con 0x7f70040cfd60 2024-04-25T16:27:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.143+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ff801d460 con 0x7f70040cfd60 2024-04-25T16:27:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.143+0000 7f7000ff9640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6ff00744b0 0x7f6ff00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.144+0000 7f7003fff640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6ff00744b0 0x7f6ff00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.144+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6ff8095640 con 0x7f70040cfd60 2024-04-25T16:27:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.144+0000 7f7003fff640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6ff00744b0 0x7f6ff00769c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70040bc330 tx=0x7f6ff4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.148+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7004148570 con 0x7f70040cfd60 2024-04-25T16:27:29.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.322+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7004159b60 con 0x7f70040cfd60 2024-04-25T16:27:29.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.322+0000 7f7000ff9640 1 -- 172.21.15.94:0/531872760 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f6ff805fa90 con 0x7f70040cfd60 2024-04-25T16:27:29.323 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:29.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6ff00744b0 msgr2=0x7f6ff00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6ff00744b0 0x7f6ff00769c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70040bc330 tx=0x7f6ff4008040 comp rx=0 tx=0).stop 2024-04-25T16:27:29.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 msgr2=0x7f70041636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 0x7f70041636c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007340 tx=0x7f6ff8007800 comp rx=0 tx=0).stop 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 shutdown_connections 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6ff00744b0 0x7f6ff00769c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7004163c00 0x7f70040baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70040cfd60 0x7f70041636c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.324+0000 7f700a9f3640 1 --2- 172.21.15.94:0/531872760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fec005250 0x7f7004148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.325+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 >> 172.21.15.94:0/531872760 conn(0x7f70040b7e90 msgr2=0x7f70040d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.325+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 shutdown_connections 2024-04-25T16:27:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.325+0000 7f700a9f3640 1 -- 172.21.15.94:0/531872760 wait complete. 2024-04-25T16:27:29.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:29.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.445+0000 7f5d407f3640 1 Processor -- start 2024-04-25T16:27:29.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.445+0000 7f5d407f3640 1 -- start start 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d407f3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d380b5460 0x7f5d380d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d407f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 0x7f5d38161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d407f3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d380d1ae0 con 0x7f5d381650b0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d407f3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5d3806cdc0 con 0x7f5d380b5460 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d407f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3806d0a0 con 0x7f5d380d16d0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d3e58f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f5d381650b0 0x7f5d3806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56368/0 (socket says 172.21.15.94:56368) 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d3e58f640 1 -- 172.21.15.94:0/3763641474 learned_addr learned my addr 172.21.15.94:0/3763641474 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d3ed90640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 0x7f5d38161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d3dd8e640 1 --2- 172.21.15.94:0/3763641474 >> v2:172.21.15.138:3300/0 conn(0x7f5d380b5460 0x7f5d380d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504805967 0 0) 0x7f5d34001490 con 0x7f5d381650b0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.447+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d24003520 con 0x7f5d381650b0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735364919 0 0) 0x7f5d24003520 con 0x7f5d381650b0 2024-04-25T16:27:29.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 msgr2=0x7f5d38161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 0x7f5d38161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 >> v2:172.21.15.138:3300/0 conn(0x7f5d380b5460 msgr2=0x7f5d380d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 --2- 172.21.15.94:0/3763641474 >> v2:172.21.15.138:3300/0 conn(0x7f5d380b5460 0x7f5d380d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3806d380 con 0x7f5d381650b0 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f5d34001be0 con 0x7f5d381650b0 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 >> v1:172.21.15.94:6789/0 conn(0x7f5d381650b0 legacy=0x7f5d3806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 0x7f5d240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d24003f30 0x7f5d24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d24005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3806cdc0 con 0x7f5d24004d40 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3806d0a0 con 0x7f5d24003f30 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2400ba10 con 0x7f5d380d16d0 2024-04-25T16:27:29.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3ed90640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d24005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3e58f640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 0x7f5d240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3dd8e640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d24003f30 0x7f5d24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3ed90640 1 -- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 msgr2=0x7f5d240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3ed90640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d380d16d0 0x7f5d240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3ed90640 1 -- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d24003f30 msgr2=0x7f5d24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3ed90640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d24003f30 0x7f5d24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.448+0000 7f5d3ed90640 1 -- 172.21.15.94:0/3763641474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d24003520 con 0x7f5d24004d40 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.449+0000 7f5d3ed90640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d24005150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d380d3fd0 tx=0x7f5d24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26a49ea368fc59c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.449+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30036070 con 0x7f5d24004d40 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.449+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30031400 con 0x7f5d24004d40 2024-04-25T16:27:29.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.449+0000 7f5d3d58d640 1 -- 172.21.15.94:0/3763641474 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30009690 con 0x7f5d24004d40 2024-04-25T16:27:29.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d407f3640 1 -- 172.21.15.94:0/3763641474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 msgr2=0x7f5d24005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d407f3640 1 --2- 172.21.15.94:0/3763641474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d24005150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d380d3fd0 tx=0x7f5d24005830 comp rx=0 tx=0).stop 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d3ed90640 1 -- 172.21.15.94:0/3763641474 reap_dead start 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d407f3640 1 -- 172.21.15.94:0/3763641474 shutdown_connections 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d407f3640 1 -- 172.21.15.94:0/3763641474 >> 172.21.15.94:0/3763641474 conn(0x7f5d380b7e90 msgr2=0x7f5d380b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d407f3640 1 -- 172.21.15.94:0/3763641474 shutdown_connections 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.450+0000 7f5d407f3640 1 -- 172.21.15.94:0/3763641474 wait complete. 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.452+0000 7f5d407f3640 1 Processor -- start 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.452+0000 7f5d407f3640 1 -- start start 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d407f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d3815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d407f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d38157d90 0x7f5d38159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d407f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d38163010 0x7f5d38163420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d407f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3806de60 con 0x7f5d24004d40 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d407f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3806d940 con 0x7f5d38163010 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d407f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d380b5870 con 0x7f5d38157d90 2024-04-25T16:27:29.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d3815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d3815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56340/0 (socket says 172.21.15.94:56340) 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 -- 172.21.15.94:0/1830525592 learned_addr learned my addr 172.21.15.94:0/1830525592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3dd8e640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d38157d90 0x7f5d38159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3ed90640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d38163010 0x7f5d38163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 -- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d38157d90 msgr2=0x7f5d38159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d38157d90 0x7f5d38159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 -- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d38163010 msgr2=0x7f5d38163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d38163010 0x7f5d38163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 -- 172.21.15.94:0/1830525592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d38163960 con 0x7f5d24004d40 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3dd8e640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d38157d90 0x7f5d38159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.453+0000 7f5d3e58f640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d3815a690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001d50 tx=0x7f5d34007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.454+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34017020 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.454+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d340133f0 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.454+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3815abd0 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.454+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340137c0 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.454+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381572b0 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.455+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3806c570 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.455+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d3401a020 con 0x7f5d24004d40 2024-04-25T16:27:29.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.456+0000 7f5d2effd640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d1c0744b0 0x7f5d1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.456+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5d34096110 con 0x7f5d24004d40 2024-04-25T16:27:29.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.456+0000 7f5d3dd8e640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d1c0744b0 0x7f5d1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.456+0000 7f5d3dd8e640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d1c0744b0 0x7f5d1c0769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d380bb250 tx=0x7f5d28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.460+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d3806c570 con 0x7f5d24004d40 2024-04-25T16:27:29.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.634+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5d3806a180 con 0x7f5d24004d40 2024-04-25T16:27:29.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.635+0000 7f5d2effd640 1 -- 172.21.15.94:0/1830525592 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f5d34021020 con 0x7f5d24004d40 2024-04-25T16:27:29.636 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d1c0744b0 msgr2=0x7f5d1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d1c0744b0 0x7f5d1c0769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d380bb250 tx=0x7f5d28008040 comp rx=0 tx=0).stop 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 msgr2=0x7f5d3815a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d3815a690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001d50 tx=0x7f5d34007430 comp rx=0 tx=0).stop 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 shutdown_connections 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d1c0744b0 0x7f5d1c0769c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d38163010 0x7f5d38163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d38157d90 0x7f5d38159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 --2- 172.21.15.94:0/1830525592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d24004d40 0x7f5d3815a690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 >> 172.21.15.94:0/1830525592 conn(0x7f5d380b7e90 msgr2=0x7f5d381616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 shutdown_connections 2024-04-25T16:27:29.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.637+0000 7f5d407f3640 1 -- 172.21.15.94:0/1830525592 wait complete. 2024-04-25T16:27:29.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:27:29.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.755+0000 7f7a31c4c640 1 Processor -- start 2024-04-25T16:27:29.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.755+0000 7f7a31c4c640 1 -- start start 2024-04-25T16:27:29.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.756+0000 7f7a31c4c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a2c160790 0x7f7a2c164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.756+0000 7f7a31c4c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7a2c06cea0 0x7f7a2c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a31c4c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a2c0b5a30 con 0x7f7a2c0b5230 2024-04-25T16:27:29.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a31c4c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2c06ca10 con 0x7f7a2c06cea0 2024-04-25T16:27:29.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a31c4c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2c0d5610 con 0x7f7a2c160790 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7a2c06cea0 0x7f7a2c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7a2c06cea0 0x7f7a2c06d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56352/0 (socket says 172.21.15.94:56352) 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 -- 172.21.15.94:0/768451628 learned_addr learned my addr 172.21.15.94:0/768451628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2affd640 1 --2- 172.21.15.94:0/768451628 >> v2:172.21.15.138:3300/0 conn(0x7f7a2c160790 0x7f7a2c164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 -- 172.21.15.94:0/768451628 >> v2:172.21.15.138:3300/0 conn(0x7f7a2c160790 msgr2=0x7f7a2c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 --2- 172.21.15.94:0/768451628 >> v2:172.21.15.138:3300/0 conn(0x7f7a2c160790 0x7f7a2c164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 -- 172.21.15.94:0/768451628 >> v1:172.21.15.138:6789/0 conn(0x7f7a2c0b5230 legacy=0x7f7a2c164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.757+0000 7f7a2bfff640 1 -- 172.21.15.94:0/768451628 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c0d58f0 con 0x7f7a2c06cea0 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.758+0000 7f7a2bfff640 1 --2- 172.21.15.94:0/768451628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c06d2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0d6130 tx=0x7f7a2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e216c0111a271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.758+0000 7f7a2a7fc640 1 -- 172.21.15.94:0/768451628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24036070 con 0x7f7a2c06cea0 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.758+0000 7f7a2a7fc640 1 -- 172.21.15.94:0/768451628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a24031600 con 0x7f7a2c06cea0 2024-04-25T16:27:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.758+0000 7f7a2a7fc640 1 -- 172.21.15.94:0/768451628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a240319d0 con 0x7f7a2c06cea0 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 -- 172.21.15.94:0/768451628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 msgr2=0x7f7a2c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/768451628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c06d2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0d6130 tx=0x7f7a2402a6a0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 -- 172.21.15.94:0/768451628 shutdown_connections 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/768451628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c06d2b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/768451628 >> v2:172.21.15.138:3300/0 conn(0x7f7a2c160790 0x7f7a2c164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 -- 172.21.15.94:0/768451628 >> 172.21.15.94:0/768451628 conn(0x7f7a2c0d3570 msgr2=0x7f7a2c0b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 -- 172.21.15.94:0/768451628 shutdown_connections 2024-04-25T16:27:29.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.759+0000 7f7a31c4c640 1 -- 172.21.15.94:0/768451628 wait complete. 2024-04-25T16:27:29.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.760+0000 7f7a31c4c640 1 Processor -- start 2024-04-25T16:27:29.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.761+0000 7f7a31c4c640 1 -- start start 2024-04-25T16:27:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a31c4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c1f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a31c4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a2c0b5230 0x7f7a2c0cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a31c4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a2c160790 0x7f7a2c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a31c4c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c0b9870 con 0x7f7a2c06cea0 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a31c4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c0b96f0 con 0x7f7a2c0b5230 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a31c4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c164e60 con 0x7f7a2c160790 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c1f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c1f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56364/0 (socket says 172.21.15.94:56364) 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 -- 172.21.15.94:0/2503514790 learned_addr learned my addr 172.21.15.94:0/2503514790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2affd640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a2c0b5230 0x7f7a2c0cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2bfff640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a2c160790 0x7f7a2c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 -- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a2c160790 msgr2=0x7f7a2c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a2c160790 0x7f7a2c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 -- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a2c0b5230 msgr2=0x7f7a2c0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a2c0b5230 0x7f7a2c0cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 -- 172.21.15.94:0/2503514790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1f7320 con 0x7f7a2c06cea0 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.762+0000 7f7a2b7fe640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c1f6a40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00c3f0 tx=0x7f7a1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.763+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0052d0 con 0x7f7a2c06cea0 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.763+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1c005490 con 0x7f7a2c06cea0 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.763+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1f7600 con 0x7f7a2c06cea0 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.763+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c007260 con 0x7f7a2c06cea0 2024-04-25T16:27:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.763+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1f7bc0 con 0x7f7a2c06cea0 2024-04-25T16:27:29.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.764+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c0b5d80 con 0x7f7a2c06cea0 2024-04-25T16:27:29.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.764+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a1c007400 con 0x7f7a2c06cea0 2024-04-25T16:27:29.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.765+0000 7f7a13fff640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7a180744e0 0x7f7a180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:29.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.765+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7a1c0948f0 con 0x7f7a2c06cea0 2024-04-25T16:27:29.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.765+0000 7f7a2affd640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7a180744e0 0x7f7a180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:29.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.765+0000 7f7a2affd640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7a180744e0 0x7f7a180769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0b4ea0 tx=0x7f7a20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:29.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.769+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a2c0b5a30 con 0x7f7a2c06cea0 2024-04-25T16:27:29.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.980+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a2c06a180 con 0x7f7a2c06cea0 2024-04-25T16:27:29.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.980+0000 7f7a13fff640 1 -- 172.21.15.94:0/2503514790 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f7a1c05e780 con 0x7f7a2c06cea0 2024-04-25T16:27:29.981 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:29.981 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:18.975101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:27:29.981 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7a180744e0 msgr2=0x7f7a180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7a180744e0 0x7f7a180769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0b4ea0 tx=0x7f7a20008040 comp rx=0 tx=0).stop 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 msgr2=0x7f7a2c1f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c1f6a40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00c3f0 tx=0x7f7a1c00c8b0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 shutdown_connections 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7a180744e0 0x7f7a180769f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a2c160790 0x7f7a2c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a2c0b5230 0x7f7a2c0cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 --2- 172.21.15.94:0/2503514790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06cea0 0x7f7a2c1f6a40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:29.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 >> 172.21.15.94:0/2503514790 conn(0x7f7a2c0d3570 msgr2=0x7f7a2c15fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:29.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.982+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 shutdown_connections 2024-04-25T16:27:29.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:29.983+0000 7f7a31c4c640 1 -- 172.21.15.94:0/2503514790 wait complete. 2024-04-25T16:27:29.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:30.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.101+0000 7fd67cb34640 1 Processor -- start 2024-04-25T16:27:30.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.101+0000 7fd67cb34640 1 -- start start 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd67cb34640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd678164a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd67cb34640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd67806cc10 0x7fd67806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd67cb34640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd678160c50 con 0x7fd6780b4dd0 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd67cb34640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd6780b5670 con 0x7fd67806cc10 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd67cb34640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd67806c780 con 0x7fd678160840 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd67806cc10 0x7fd67806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd67806cc10 0x7fd67806d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56368/0 (socket says 172.21.15.94:56368) 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 -- 172.21.15.94:0/4238609598 learned_addr learned my addr 172.21.15.94:0/4238609598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd675d9b640 1 --2- 172.21.15.94:0/4238609598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd678164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 -- 172.21.15.94:0/4238609598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 msgr2=0x7fd678164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 --2- 172.21.15.94:0/4238609598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd678164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 -- 172.21.15.94:0/4238609598 >> v1:172.21.15.138:6789/0 conn(0x7fd6780b4dd0 legacy=0x7fd678164330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.103+0000 7fd676d9d640 1 -- 172.21.15.94:0/4238609598 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780d57b0 con 0x7fd67806cc10 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.104+0000 7fd676d9d640 1 --2- 172.21.15.94:0/4238609598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd67806d020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd670009870 tx=0x7fd67002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af891ccce761fb81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.104+0000 7fd67559a640 1 -- 172.21.15.94:0/4238609598 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670037070 con 0x7fd67806cc10 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.104+0000 7fd67559a640 1 -- 172.21.15.94:0/4238609598 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670032440 con 0x7fd67806cc10 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.104+0000 7fd67559a640 1 -- 172.21.15.94:0/4238609598 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670032810 con 0x7fd67806cc10 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 -- 172.21.15.94:0/4238609598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 msgr2=0x7fd67806d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 --2- 172.21.15.94:0/4238609598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd67806d020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd670009870 tx=0x7fd67002a180 comp rx=0 tx=0).stop 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 -- 172.21.15.94:0/4238609598 shutdown_connections 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 --2- 172.21.15.94:0/4238609598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd67806d020 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 --2- 172.21.15.94:0/4238609598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd678164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 -- 172.21.15.94:0/4238609598 >> 172.21.15.94:0/4238609598 conn(0x7fd6780d6200 msgr2=0x7fd6780d6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 -- 172.21.15.94:0/4238609598 shutdown_connections 2024-04-25T16:27:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.105+0000 7fd67cb34640 1 -- 172.21.15.94:0/4238609598 wait complete. 2024-04-25T16:27:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.107+0000 7fd67cb34640 1 Processor -- start 2024-04-25T16:27:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.107+0000 7fd67cb34640 1 -- start start 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67cb34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd6780b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67cb34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6780b4dd0 0x7fd6780cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67cb34640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd6781fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67cb34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd678164f80 con 0x7fd67806cc10 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67cb34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780b5320 con 0x7fd6780b4dd0 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67cb34640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd678165100 con 0x7fd678160840 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd6780b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd6780b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56378/0 (socket says 172.21.15.94:56378) 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 -- 172.21.15.94:0/2698124066 learned_addr learned my addr 172.21.15.94:0/2698124066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd676d9d640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd6781fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd675d9b640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6780b4dd0 0x7fd6780cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 -- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 msgr2=0x7fd6781fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd6781fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 -- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6780b4dd0 msgr2=0x7fd6780cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6780b4dd0 0x7fd6780cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.108+0000 7fd67659c640 1 -- 172.21.15.94:0/2698124066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780b76e0 con 0x7fd67806cc10 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.109+0000 7fd67659c640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd6780b5c60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b9f0 tx=0x7fd66800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.109+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66800cbd0 con 0x7fd67806cc10 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.109+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66800cd70 con 0x7fd67806cc10 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.109+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6780b6640 con 0x7fd67806cc10 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.109+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668003bb0 con 0x7fd67806cc10 2024-04-25T16:27:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.109+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6780b6c00 con 0x7fd67806cc10 2024-04-25T16:27:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.110+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67806a180 con 0x7fd67806cc10 2024-04-25T16:27:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.111+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd668014440 con 0x7fd67806cc10 2024-04-25T16:27:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.111+0000 7fd666ffd640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6580744b0 0x7fd6580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.111+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6680954a0 con 0x7fd67806cc10 2024-04-25T16:27:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.111+0000 7fd675d9b640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6580744b0 0x7fd6580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.112+0000 7fd675d9b640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6580744b0 0x7fd6580769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd66c006fd0 tx=0x7fd66c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.116+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd66805f8f0 con 0x7fd67806cc10 2024-04-25T16:27:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.289+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6780d4700 con 0x7fd67806cc10 2024-04-25T16:27:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.290+0000 7fd666ffd640 1 -- 172.21.15.94:0/2698124066 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fd66805f1f0 con 0x7fd67806cc10 2024-04-25T16:27:30.291 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:30.291 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:30.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6580744b0 msgr2=0x7fd6580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6580744b0 0x7fd6580769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd66c006fd0 tx=0x7fd66c008040 comp rx=0 tx=0).stop 2024-04-25T16:27:30.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 msgr2=0x7fd6780b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd6780b5c60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b9f0 tx=0x7fd66800beb0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 shutdown_connections 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6580744b0 0x7fd6580769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd678160840 0x7fd6781fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6780b4dd0 0x7fd6780cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 --2- 172.21.15.94:0/2698124066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd67806cc10 0x7fd6780b5c60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 >> 172.21.15.94:0/2698124066 conn(0x7fd6780d6200 msgr2=0x7fd6780bb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 shutdown_connections 2024-04-25T16:27:30.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.292+0000 7fd67cb34640 1 -- 172.21.15.94:0/2698124066 wait complete. 2024-04-25T16:27:30.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:30.305 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.415+0000 7fd6f0484640 1 Processor -- start 2024-04-25T16:27:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.415+0000 7fd6f0484640 1 -- start start 2024-04-25T16:27:30.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.416+0000 7fd6f0484640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd6e80b97b0 0x7fd6e80d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6f0484640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e80cf6d0 con 0x7fd6e80b7440 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6f0484640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e80cf9b0 con 0x7fd6e80d3640 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6f0484640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e80cfc90 con 0x7fd6e80b97b0 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6ee220640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd6e80b97b0 0x7fd6e80d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6eea21640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd6e80b7440 0x7fd6e80b7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56374/0 (socket says 172.21.15.94:56374) 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6ee220640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd6e80b97b0 0x7fd6e80d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56384/0 (socket says 172.21.15.94:56384) 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6eea21640 1 -- 172.21.15.94:0/3424403283 learned_addr learned my addr 172.21.15.94:0/3424403283 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6ee220640 1 -- 172.21.15.94:0/3424403283 >> v1:172.21.15.138:6789/0 conn(0x7fd6e80d3640 legacy=0x7fd6e80cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6ee220640 1 -- 172.21.15.94:0/3424403283 >> v1:172.21.15.94:6789/0 conn(0x7fd6e80b7440 legacy=0x7fd6e80b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6ee220640 1 -- 172.21.15.94:0/3424403283 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e80d2900 con 0x7fd6e80b97b0 2024-04-25T16:27:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.417+0000 7fd6ed21e640 1 -- 172.21.15.94:0/3424403283 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011477047 0 0) 0x7fd6e80cf6d0 con 0x7fd6e80b7440 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.418+0000 7fd6ee220640 1 --2- 172.21.15.94:0/3424403283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80b97b0 0x7fd6e80d3100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40097a0 tx=0x7fd6e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d1c2e1a75e0b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.418+0000 7fd6ed21e640 1 -- 172.21.15.94:0/3424403283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4037070 con 0x7fd6e80b97b0 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.418+0000 7fd6ed21e640 1 -- 172.21.15.94:0/3424403283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e4032440 con 0x7fd6e80b97b0 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.418+0000 7fd6ed21e640 1 -- 172.21.15.94:0/3424403283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4032810 con 0x7fd6e80b97b0 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.418+0000 7fd6f0484640 1 -- 172.21.15.94:0/3424403283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80b97b0 msgr2=0x7fd6e80d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.418+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3424403283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80b97b0 0x7fd6e80d3100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40097a0 tx=0x7fd6e402a230 comp rx=0 tx=0).stop 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.419+0000 7fd6f0484640 1 -- 172.21.15.94:0/3424403283 shutdown_connections 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.419+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3424403283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80b97b0 0x7fd6e80d3100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.419+0000 7fd6f0484640 1 -- 172.21.15.94:0/3424403283 >> 172.21.15.94:0/3424403283 conn(0x7fd6e80d3fa0 msgr2=0x7fd6e80d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.419+0000 7fd6f0484640 1 -- 172.21.15.94:0/3424403283 shutdown_connections 2024-04-25T16:27:30.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.419+0000 7fd6f0484640 1 -- 172.21.15.94:0/3424403283 wait complete. 2024-04-25T16:27:30.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.420+0000 7fd6f0484640 1 Processor -- start 2024-04-25T16:27:30.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.420+0000 7fd6f0484640 1 -- start start 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6f0484640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e80b7440 0x7fd6e80cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6f0484640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b97b0 0x7fd6e80aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6f0484640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 0x7fd6e815d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6f0484640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8162360 con 0x7fd6e80d3640 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6f0484640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e80b9bc0 con 0x7fd6e80b7440 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6f0484640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e80cf340 con 0x7fd6e80b97b0 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 0x7fd6e815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 0x7fd6e815d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56394/0 (socket says 172.21.15.94:56394) 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 -- 172.21.15.94:0/3672473182 learned_addr learned my addr 172.21.15.94:0/3672473182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6ee220640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e80b7440 0x7fd6e80cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eda1f640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b97b0 0x7fd6e80aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 -- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b97b0 msgr2=0x7fd6e80aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b97b0 0x7fd6e80aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 -- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e80b7440 msgr2=0x7fd6e80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e80b7440 0x7fd6e80cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 -- 172.21.15.94:0/3672473182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e80abf40 con 0x7fd6e80d3640 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6ee220640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e80b7440 0x7fd6e80cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6eea21640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 0x7fd6e815d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c700 tx=0x7fd6e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.422+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0018070 con 0x7fd6e80d3640 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.423+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e000ce10 con 0x7fd6e80d3640 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.423+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000a860 con 0x7fd6e80d3640 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.423+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e80ccf50 con 0x7fd6e80d3640 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.423+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e80cd510 con 0x7fd6e80d3640 2024-04-25T16:27:30.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.423+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e806a180 con 0x7fd6e80d3640 2024-04-25T16:27:30.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.424+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6e001b070 con 0x7fd6e80d3640 2024-04-25T16:27:30.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.425+0000 7fd6deffd640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6cc0744e0 0x7fd6cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.425+0000 7fd6ee220640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6cc0744e0 0x7fd6cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.425+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6e0094ab0 con 0x7fd6e80d3640 2024-04-25T16:27:30.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.425+0000 7fd6ee220640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6cc0744e0 0x7fd6cc0769f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4034000 tx=0x7fd6e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.429+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6e005ef00 con 0x7fd6e80d3640 2024-04-25T16:27:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.607+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6e80d2380 con 0x7fd6e80d3640 2024-04-25T16:27:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.607+0000 7fd6deffd640 1 -- 172.21.15.94:0/3672473182 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd6e005e800 con 0x7fd6e80d3640 2024-04-25T16:27:30.608 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6cc0744e0 msgr2=0x7fd6cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6cc0744e0 0x7fd6cc0769f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4034000 tx=0x7fd6e402e000 comp rx=0 tx=0).stop 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 msgr2=0x7fd6e815d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 0x7fd6e815d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c700 tx=0x7fd6e000cbc0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 shutdown_connections 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd6cc0744e0 0x7fd6cc0769f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d3640 0x7fd6e815d420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b97b0 0x7fd6e80aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 --2- 172.21.15.94:0/3672473182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e80b7440 0x7fd6e80cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 >> 172.21.15.94:0/3672473182 conn(0x7fd6e80d3fa0 msgr2=0x7fd6e80b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 shutdown_connections 2024-04-25T16:27:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.609+0000 7fd6f0484640 1 -- 172.21.15.94:0/3672473182 wait complete. 2024-04-25T16:27:30.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.731+0000 7f716b0d8640 1 Processor -- start 2024-04-25T16:27:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.731+0000 7f716b0d8640 1 -- start start 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f716b0d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 0x7f716409c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f716b0d8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f716409c7f0 0x7f7164162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f716b0d8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640d2270 con 0x7f7164163280 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f716b0d8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f71640b34b0 con 0x7f716409c7f0 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f716b0d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71640d1f90 con 0x7f71640b5420 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f7169675640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7164163280 0x7f71640ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56384/0 (socket says 172.21.15.94:56384) 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f7169675640 1 -- 172.21.15.94:0/4037640355 learned_addr learned my addr 172.21.15.94:0/4037640355 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f7163fff640 1 --2- 172.21.15.94:0/4037640355 >> v2:172.21.15.138:3300/0 conn(0x7f716409c7f0 0x7f7164162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f7168e74640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 0x7f716409c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571472614 0 0) 0x7f71640d2270 con 0x7f7164163280 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f714c003520 con 0x7f7164163280 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3865613475 0 0) 0x7f714c003520 con 0x7f7164163280 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 msgr2=0x7f716409c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 0x7f716409c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 >> v2:172.21.15.138:3300/0 conn(0x7f716409c7f0 msgr2=0x7f7164162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 --2- 172.21.15.94:0/4037640355 >> v2:172.21.15.138:3300/0 conn(0x7f716409c7f0 0x7f7164162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716409d150 con 0x7f7164163280 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f715c0019f0 con 0x7f7164163280 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 >> v1:172.21.15.94:6789/0 conn(0x7f7164163280 legacy=0x7f71640ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f714c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c004800 0x7f714c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 0x7f714c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71640b34b0 con 0x7f714c0039b0 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71640d1f90 con 0x7f714c004800 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f714c00ba70 con 0x7f71640b5420 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.733+0000 7f7168e74640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f714c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7169675640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 0x7f714c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7163fff640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c004800 0x7f714c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7168e74640 1 -- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 msgr2=0x7f714c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7168e74640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71640b5420 0x7f714c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7168e74640 1 -- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c004800 msgr2=0x7f714c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7168e74640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c004800 0x7f714c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7168e74640 1 -- 172.21.15.94:0/4037640355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c003520 con 0x7f714c0039b0 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7163fff640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c004800 0x7f714c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.734+0000 7f7168e74640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f714c003e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7154009ac0 tx=0x7f714c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff9b03ba35f2bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154036070 con 0x7f714c0039b0 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7154031400 con 0x7f714c0039b0 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f71637fe640 1 -- 172.21.15.94:0/4037640355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154009690 con 0x7f714c0039b0 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f716b0d8640 1 -- 172.21.15.94:0/4037640355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 msgr2=0x7f714c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f716b0d8640 1 --2- 172.21.15.94:0/4037640355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f714c003e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7154009ac0 tx=0x7f714c005830 comp rx=0 tx=0).stop 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f7169675640 1 -- 172.21.15.94:0/4037640355 reap_dead start 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f716b0d8640 1 -- 172.21.15.94:0/4037640355 shutdown_connections 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f716b0d8640 1 -- 172.21.15.94:0/4037640355 >> 172.21.15.94:0/4037640355 conn(0x7f71640b5b20 msgr2=0x7f71640b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f716b0d8640 1 -- 172.21.15.94:0/4037640355 shutdown_connections 2024-04-25T16:27:30.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.735+0000 7f716b0d8640 1 -- 172.21.15.94:0/4037640355 wait complete. 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.737+0000 7f716b0d8640 1 Processor -- start 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.737+0000 7f716b0d8640 1 -- start start 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.738+0000 7f716b0d8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f71640a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.738+0000 7f716b0d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71641570a0 0x7f71640cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.738+0000 7f716b0d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7164151000 0x7f71641f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.738+0000 7f716b0d8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71640b0c20 con 0x7f714c0039b0 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.738+0000 7f716b0d8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71640cfed0 con 0x7f7164151000 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.738+0000 7f716b0d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71640b0da0 con 0x7f71641570a0 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f71640a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f71640a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56424/0 (socket says 172.21.15.94:56424) 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 -- 172.21.15.94:0/601680836 learned_addr learned my addr 172.21.15.94:0/601680836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7163fff640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71641570a0 0x7f71640cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7169675640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7164151000 0x7f71641f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 -- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71641570a0 msgr2=0x7f71640cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71641570a0 0x7f71640cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 -- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7164151000 msgr2=0x7f71641f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7164151000 0x7f71641f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 -- 172.21.15.94:0/601680836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71640ca500 con 0x7f714c0039b0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7169675640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7164151000 0x7f71641f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f7168e74640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f71640a1810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7154009be0 tx=0x7f7154004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154040020 con 0x7f714c0039b0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7154003b40 con 0x7f714c0039b0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71640ca7e0 con 0x7f714c0039b0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154036040 con 0x7f714c0039b0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.739+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71640ccfc0 con 0x7f714c0039b0 2024-04-25T16:27:30.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.740+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716406a180 con 0x7f714c0039b0 2024-04-25T16:27:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.741+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7154043020 con 0x7f714c0039b0 2024-04-25T16:27:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.741+0000 7f71617fa640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f71500744b0 0x7f71500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.742+0000 7f7163fff640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f71500744b0 0x7f71500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.742+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f71540b5820 con 0x7f714c0039b0 2024-04-25T16:27:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.742+0000 7f7163fff640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f71500744b0 0x7f71500769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f71580090e0 tx=0x7f7158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.746+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f715404a070 con 0x7f714c0039b0 2024-04-25T16:27:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.921+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71640ca9b0 con 0x7f714c0039b0 2024-04-25T16:27:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.921+0000 7f71617fa640 1 -- 172.21.15.94:0/601680836 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f71540044e0 con 0x7f714c0039b0 2024-04-25T16:27:30.922 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.923+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f71500744b0 msgr2=0x7f71500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.923+0000 7f716b0d8640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f71500744b0 0x7f71500769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f71580090e0 tx=0x7f7158008040 comp rx=0 tx=0).stop 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.923+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 msgr2=0x7f71640a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.923+0000 7f716b0d8640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f71640a1810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7154009be0 tx=0x7f7154004730 comp rx=0 tx=0).stop 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 shutdown_connections 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f71500744b0 0x7f71500769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7164151000 0x7f71641f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71641570a0 0x7f71640cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 --2- 172.21.15.94:0/601680836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c0039b0 0x7f71640a1810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 >> 172.21.15.94:0/601680836 conn(0x7f71640b5b20 msgr2=0x7f71640b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 shutdown_connections 2024-04-25T16:27:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:30.924+0000 7f716b0d8640 1 -- 172.21.15.94:0/601680836 wait complete. 2024-04-25T16:27:30.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:31.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.045+0000 7fc2a70a1640 1 Processor -- start 2024-04-25T16:27:31.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.045+0000 7fc2a70a1640 1 -- start start 2024-04-25T16:27:31.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc2a70a1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2a0164e40 0x7fc2a0165250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc2a70a1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a00d39e0 con 0x7fc2a0165790 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc2a70a1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a00d2dd0 con 0x7fc2a00b9780 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc2a70a1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a00d30b0 con 0x7fc2a0164e40 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc2a563e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc2a0165790 0x7fc2a00ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56394/0 (socket says 172.21.15.94:56394) 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc2a563e640 1 -- 172.21.15.94:0/3527678813 learned_addr learned my addr 172.21.15.94:0/3527678813 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc29ffff640 1 --2- 172.21.15.94:0/3527678813 >> v2:172.21.15.138:3300/0 conn(0x7fc2a0164e40 0x7fc2a0165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276437139 0 0) 0x7fc2a00d39e0 con 0x7fc2a0165790 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.046+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc288003520 con 0x7fc2a0165790 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.047+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142740395 0 0) 0x7fc2a00d2dd0 con 0x7fc2a00b9780 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.047+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2a00d39e0 con 0x7fc2a00b9780 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.047+0000 7fc29ffff640 1 -- 172.21.15.94:0/3527678813 >> v1:172.21.15.138:6789/0 conn(0x7fc2a00b9780 legacy=0x7fc2a009c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.047+0000 7fc29ffff640 1 -- 172.21.15.94:0/3527678813 >> v1:172.21.15.94:6789/0 conn(0x7fc2a0165790 legacy=0x7fc2a00ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.047+0000 7fc29ffff640 1 -- 172.21.15.94:0/3527678813 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a00d3390 con 0x7fc2a0164e40 2024-04-25T16:27:31.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.047+0000 7fc29ffff640 1 --2- 172.21.15.94:0/3527678813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0164e40 0x7fc2a0165250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2940097a0 tx=0x7fc29402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54021eb55e555f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294037070 con 0x7fc2a0164e40 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc294032440 con 0x7fc2a0164e40 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc29f7fe640 1 -- 172.21.15.94:0/3527678813 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294032810 con 0x7fc2a0164e40 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 -- 172.21.15.94:0/3527678813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0164e40 msgr2=0x7fc2a0165250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/3527678813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0164e40 0x7fc2a0165250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2940097a0 tx=0x7fc29402a230 comp rx=0 tx=0).stop 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 -- 172.21.15.94:0/3527678813 shutdown_connections 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/3527678813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0164e40 0x7fc2a0165250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 -- 172.21.15.94:0/3527678813 >> 172.21.15.94:0/3527678813 conn(0x7fc2a00b5b20 msgr2=0x7fc2a00b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 -- 172.21.15.94:0/3527678813 shutdown_connections 2024-04-25T16:27:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.048+0000 7fc2a70a1640 1 -- 172.21.15.94:0/3527678813 wait complete. 2024-04-25T16:27:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.050+0000 7fc2a70a1640 1 Processor -- start 2024-04-25T16:27:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.050+0000 7fc2a70a1640 1 -- start start 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.051+0000 7fc2a70a1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2a00b9780 0x7fc2a00cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 0x7fc2a01f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0165790 0x7fc2a015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0165ba0 con 0x7fc2a0164e40 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a009c9c0 con 0x7fc2a0165790 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a00b4f60 con 0x7fc2a00b9780 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 0x7fc2a01f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 0x7fc2a01f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56428/0 (socket says 172.21.15.94:56428) 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 -- 172.21.15.94:0/2212323704 learned_addr learned my addr 172.21.15.94:0/2212323704 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a563e640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0165790 0x7fc2a015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a4e3d640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2a00b9780 0x7fc2a00cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 -- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2a00b9780 msgr2=0x7fc2a00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2a00b9780 0x7fc2a00cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 -- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0165790 msgr2=0x7fc2a015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0165790 0x7fc2a015c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 -- 172.21.15.94:0/2212323704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a00c7db0 con 0x7fc2a0164e40 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29ffff640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 0x7fc2a01f7360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc294035040 tx=0x7fc2940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294041030 con 0x7fc2a0164e40 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29403d4e0 con 0x7fc2a0164e40 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a00c8090 con 0x7fc2a0164e40 2024-04-25T16:27:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294037040 con 0x7fc2a0164e40 2024-04-25T16:27:31.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.052+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a00c8650 con 0x7fc2a0164e40 2024-04-25T16:27:31.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.053+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a006fe00 con 0x7fc2a0164e40 2024-04-25T16:27:31.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.054+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc294044070 con 0x7fc2a0164e40 2024-04-25T16:27:31.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.054+0000 7fc29d7fa640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc28c0744e0 0x7fc28c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.055+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc2940b5b60 con 0x7fc2a0164e40 2024-04-25T16:27:31.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.055+0000 7fc2a4e3d640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc28c0744e0 0x7fc28c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.055+0000 7fc2a4e3d640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc28c0744e0 0x7fc28c0769f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc290007fd0 tx=0x7fc290009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.059+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2a006fe00 con 0x7fc2a0164e40 2024-04-25T16:27:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.235+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2a00cdbe0 con 0x7fc2a0164e40 2024-04-25T16:27:31.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.236+0000 7fc29d7fa640 1 -- 172.21.15.94:0/2212323704 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc29407f970 con 0x7fc2a0164e40 2024-04-25T16:27:31.236 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc28c0744e0 msgr2=0x7fc28c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc28c0744e0 0x7fc28c0769f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc290007fd0 tx=0x7fc290009040 comp rx=0 tx=0).stop 2024-04-25T16:27:31.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 msgr2=0x7fc2a01f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 0x7fc2a01f7360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc294035040 tx=0x7fc2940046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 shutdown_connections 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc28c0744e0 0x7fc28c0769f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0165790 0x7fc2a015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a0164e40 0x7fc2a01f7360 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 --2- 172.21.15.94:0/2212323704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2a00b9780 0x7fc2a00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 >> 172.21.15.94:0/2212323704 conn(0x7fc2a00b5b20 msgr2=0x7fc2a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 shutdown_connections 2024-04-25T16:27:31.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.238+0000 7fc2a70a1640 1 -- 172.21.15.94:0/2212323704 wait complete. 2024-04-25T16:27:31.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.360+0000 7f7e2cfce640 1 Processor -- start 2024-04-25T16:27:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.360+0000 7f7e2cfce640 1 -- start start 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e2cfce640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7e280d3890 0x7f7e280b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e2cfce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e280d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e2cfce640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2806f840 con 0x7f7e280cea30 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e2cfce640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7e280d0830 con 0x7f7e280d3890 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e2cfce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e280b2810 con 0x7f7e280cfcf0 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e2659c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7e280d3890 0x7f7e280b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e26d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7e280cea30 0x7f7e280cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56398/0 (socket says 172.21.15.94:56398) 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e26d9d640 1 -- 172.21.15.94:0/853780536 learned_addr learned my addr 172.21.15.94:0/853780536 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.361+0000 7f7e25d9b640 1 --2- 172.21.15.94:0/853780536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e280d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2659c640 1 -- 172.21.15.94:0/853780536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 msgr2=0x7f7e280d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2659c640 1 --2- 172.21.15.94:0/853780536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e280d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2659c640 1 -- 172.21.15.94:0/853780536 >> v1:172.21.15.94:6789/0 conn(0x7f7e280cea30 legacy=0x7f7e280cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2659c640 1 -- 172.21.15.94:0/853780536 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e280b2af0 con 0x7f7e280d3890 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2659c640 1 --2- 172.21.15.94:0/853780536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280d3890 0x7f7e280b38b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009de0 tx=0x7f7e1802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a8181968ee65d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2559a640 1 -- 172.21.15.94:0/853780536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18007740 con 0x7f7e280d3890 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2559a640 1 -- 172.21.15.94:0/853780536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e180078e0 con 0x7f7e280d3890 2024-04-25T16:27:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.362+0000 7f7e2559a640 1 -- 172.21.15.94:0/853780536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18007cb0 con 0x7f7e280d3890 2024-04-25T16:27:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 -- 172.21.15.94:0/853780536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280d3890 msgr2=0x7f7e280b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/853780536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280d3890 0x7f7e280b38b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009de0 tx=0x7f7e1802a6c0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 -- 172.21.15.94:0/853780536 shutdown_connections 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/853780536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e280d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/853780536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280d3890 0x7f7e280b38b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 -- 172.21.15.94:0/853780536 >> 172.21.15.94:0/853780536 conn(0x7f7e280b7b20 msgr2=0x7f7e280b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 -- 172.21.15.94:0/853780536 shutdown_connections 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.363+0000 7f7e2cfce640 1 -- 172.21.15.94:0/853780536 wait complete. 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.365+0000 7f7e2cfce640 1 Processor -- start 2024-04-25T16:27:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.365+0000 7f7e2cfce640 1 -- start start 2024-04-25T16:27:31.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.366+0000 7f7e2cfce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 0x7f7e281578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.366+0000 7f7e2cfce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e281f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2cfce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e280d3890 0x7f7e28153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2cfce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e280b3390 con 0x7f7e280cea30 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2cfce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e280b30b0 con 0x7f7e280d3890 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2cfce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e280b3f80 con 0x7f7e280cfcf0 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 0x7f7e281578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 0x7f7e281578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56448/0 (socket says 172.21.15.94:56448) 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 -- 172.21.15.94:0/4269699688 learned_addr learned my addr 172.21.15.94:0/4269699688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e26d9d640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e280d3890 0x7f7e28153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e25d9b640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e281f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 -- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 msgr2=0x7f7e281f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e281f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 -- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e280d3890 msgr2=0x7f7e28153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e280d3890 0x7f7e28153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 -- 172.21.15.94:0/4269699688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e28154110 con 0x7f7e280cea30 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2659c640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 0x7f7e281578d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009f00 tx=0x7f7e18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1803f070 con 0x7f7e280cea30 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18031c40 con 0x7f7e280cea30 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18044460 con 0x7f7e280cea30 2024-04-25T16:27:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e280b1e80 con 0x7f7e280cea30 2024-04-25T16:27:31.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.367+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281f4900 con 0x7f7e280cea30 2024-04-25T16:27:31.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.368+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2806a180 con 0x7f7e280cea30 2024-04-25T16:27:31.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.369+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e18044600 con 0x7f7e280cea30 2024-04-25T16:27:31.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.369+0000 7f7e16ffd640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7e080744b0 0x7f7e080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.369+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e180b6c90 con 0x7f7e280cea30 2024-04-25T16:27:31.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.369+0000 7f7e25d9b640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7e080744b0 0x7f7e080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.370+0000 7f7e25d9b640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7e080744b0 0x7f7e080769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c006fd0 tx=0x7f7e1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.374+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e18081170 con 0x7f7e280cea30 2024-04-25T16:27:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.549+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7e280cfad0 con 0x7f7e280cea30 2024-04-25T16:27:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.550+0000 7f7e16ffd640 1 -- 172.21.15.94:0/4269699688 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f7e18080a70 con 0x7f7e280cea30 2024-04-25T16:27:31.551 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7e080744b0 msgr2=0x7f7e080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7e080744b0 0x7f7e080769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c006fd0 tx=0x7f7e1c008040 comp rx=0 tx=0).stop 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 msgr2=0x7f7e281578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 0x7f7e281578d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009f00 tx=0x7f7e18004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 shutdown_connections 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7e080744b0 0x7f7e080769c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e280d3890 0x7f7e28153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e280cfcf0 0x7f7e281f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 --2- 172.21.15.94:0/4269699688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e280cea30 0x7f7e281578d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 >> 172.21.15.94:0/4269699688 conn(0x7f7e280b7b20 msgr2=0x7f7e280afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 shutdown_connections 2024-04-25T16:27:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.552+0000 7f7e2cfce640 1 -- 172.21.15.94:0/4269699688 wait complete. 2024-04-25T16:27:31.564 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:27:31.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.670+0000 7fe8213f8640 1 Processor -- start 2024-04-25T16:27:31.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.670+0000 7fe8213f8640 1 -- start start 2024-04-25T16:27:31.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe8213f8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe81c0d3480 0x7fe81c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe8213f8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81c06d500 con 0x7fe81c145240 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe8213f8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81c162190 con 0x7fe81c06f4c0 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe8213f8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe81c06f8d0 con 0x7fe81c0d3480 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe81c145240 0x7fe81c145650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46670/0 (socket says 172.21.15.94:46670) 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81b7fe640 1 -- 172.21.15.94:0/2182017584 learned_addr learned my addr 172.21.15.94:0/2182017584 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe813fff640 1 --2- 172.21.15.94:0/2182017584 >> v2:172.21.15.138:3300/0 conn(0x7fe81c0d3480 0x7fe81c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559676476 0 0) 0x7fe81c06d500 con 0x7fe81c145240 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe808003520 con 0x7fe81c145240 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1959940241 0 0) 0x7fe808003520 con 0x7fe81c145240 2024-04-25T16:27:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 >> v2:172.21.15.138:3300/0 conn(0x7fe81c0d3480 msgr2=0x7fe81c06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 --2- 172.21.15.94:0/2182017584 >> v2:172.21.15.138:3300/0 conn(0x7fe81c0d3480 0x7fe81c06ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 >> v1:172.21.15.138:6789/0 conn(0x7fe81c06f4c0 legacy=0x7fe81c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c162470 con 0x7fe81c145240 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fe8140019f0 con 0x7fe81c145240 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 >> v1:172.21.15.94:6789/0 conn(0x7fe81c145240 legacy=0x7fe81c145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8080039b0 0x7fe808003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe808004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8080051c0 0x7fe80800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c06f8d0 con 0x7fe808004800 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe80800c160 con 0x7fe8080051c0 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe80800c360 con 0x7fe8080039b0 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.672+0000 7fe813fff640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe808004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe81affd640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8080039b0 0x7fe808003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe81b7fe640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8080051c0 0x7fe80800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe813fff640 1 -- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8080039b0 msgr2=0x7fe808003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe813fff640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8080039b0 0x7fe808003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe813fff640 1 -- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8080051c0 msgr2=0x7fe80800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe813fff640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8080051c0 0x7fe80800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe813fff640 1 -- 172.21.15.94:0/2182017584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe808003520 con 0x7fe808004800 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe81b7fe640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8080051c0 0x7fe80800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe81b7fe640 1 -- 172.21.15.94:0/2182017584 reap_dead start 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.673+0000 7fe813fff640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe808004c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe804009ac0 tx=0x7fe808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffa332806845bea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804036070 con 0x7fe808004800 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe804031400 con 0x7fe808004800 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe81a7fc640 1 -- 172.21.15.94:0/2182017584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804009690 con 0x7fe808004800 2024-04-25T16:27:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 -- 172.21.15.94:0/2182017584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 msgr2=0x7fe808004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe808004c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe804009ac0 tx=0x7fe808005830 comp rx=0 tx=0).stop 2024-04-25T16:27:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 -- 172.21.15.94:0/2182017584 shutdown_connections 2024-04-25T16:27:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2182017584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe808004c10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 -- 172.21.15.94:0/2182017584 >> 172.21.15.94:0/2182017584 conn(0x7fe81c0b7cc0 msgr2=0x7fe81c0b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 -- 172.21.15.94:0/2182017584 shutdown_connections 2024-04-25T16:27:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.674+0000 7fe8213f8640 1 -- 172.21.15.94:0/2182017584 wait complete. 2024-04-25T16:27:31.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.676+0000 7fe8213f8640 1 Processor -- start 2024-04-25T16:27:31.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.676+0000 7fe8213f8640 1 -- start start 2024-04-25T16:27:31.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.677+0000 7fe8213f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe81c15eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c1f45b0 0x7fe81c09d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c1f49c0 0x7fe81c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c161c70 con 0x7fe808004800 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c162ac0 con 0x7fe81c1f45b0 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe81c15fe50 con 0x7fe81c1f49c0 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe81c15eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe81c15eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59090/0 (socket says 172.21.15.94:59090) 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 -- 172.21.15.94:0/2849135304 learned_addr learned my addr 172.21.15.94:0/2849135304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe813fff640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c1f45b0 0x7fe81c09d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81b7fe640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c1f49c0 0x7fe81c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 -- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c1f49c0 msgr2=0x7fe81c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c1f49c0 0x7fe81c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 -- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c1f45b0 msgr2=0x7fe81c09d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c1f45b0 0x7fe81c09d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 -- 172.21.15.94:0/2849135304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c1f5470 con 0x7fe808004800 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81b7fe640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c1f49c0 0x7fe81c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe81affd640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe81c15eaa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00f820 tx=0x7fe80c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c001ce0 con 0x7fe808004800 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c018070 con 0x7fe808004800 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00d730 con 0x7fe808004800 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c15d6a0 con 0x7fe808004800 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.678+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c15dfc0 con 0x7fe808004800 2024-04-25T16:27:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.680+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c06a180 con 0x7fe808004800 2024-04-25T16:27:31.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.680+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe80c00d8d0 con 0x7fe808004800 2024-04-25T16:27:31.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.680+0000 7fe8137fe640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe7fc0744b0 0x7fe7fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:31.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.680+0000 7fe813fff640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe7fc0744b0 0x7fe7fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:31.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.681+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe80c095340 con 0x7fe808004800 2024-04-25T16:27:31.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.681+0000 7fe813fff640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe7fc0744b0 0x7fe7fc0769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe804033000 tx=0x7fe804008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:31.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.685+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe80c05f790 con 0x7fe808004800 2024-04-25T16:27:31.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.899+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe81c071380 con 0x7fe808004800 2024-04-25T16:27:31.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.899+0000 7fe8137fe640 1 -- 172.21.15.94:0/2849135304 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fe80c05f090 con 0x7fe808004800 2024-04-25T16:27:31.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:31.900 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:18.975101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:27:31.901 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:27:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe7fc0744b0 msgr2=0x7fe7fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe7fc0744b0 0x7fe7fc0769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe804033000 tx=0x7fe804008030 comp rx=0 tx=0).stop 2024-04-25T16:27:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 msgr2=0x7fe81c15eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe81c15eaa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00f820 tx=0x7fe80c00fce0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 shutdown_connections 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe7fc0744b0 0x7fe7fc0769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c1f49c0 0x7fe81c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c1f45b0 0x7fe81c09d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 --2- 172.21.15.94:0/2849135304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe808004800 0x7fe81c15eaa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 >> 172.21.15.94:0/2849135304 conn(0x7fe81c0b7cc0 msgr2=0x7fe81c070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 shutdown_connections 2024-04-25T16:27:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:31.901+0000 7fe8213f8640 1 -- 172.21.15.94:0/2849135304 wait complete. 2024-04-25T16:27:31.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.020+0000 7fdeca00c640 1 Processor -- start 2024-04-25T16:27:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.020+0000 7fdeca00c640 1 -- start start 2024-04-25T16:27:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.021+0000 7fdeca00c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec40b5230 0x7fdec4160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.021+0000 7fdeca00c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdec4164210 0x7fdec4164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.021+0000 7fdeca00c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec40b5a30 con 0x7fdec4164b60 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.021+0000 7fdeca00c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdec406ca10 con 0x7fdec4164210 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.021+0000 7fdeca00c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdec40d5610 con 0x7fdec40b5230 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec2ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdec4164210 0x7fdec4164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec3fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdec4164b60 0x7fdec40d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:56680/0 (socket says 172.21.15.94:56680) 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec3fff640 1 -- 172.21.15.94:0/2607989109 learned_addr learned my addr 172.21.15.94:0/2607989109 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec37fe640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec40b5230 0x7fdec4160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec2ffd640 1 -- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec40b5230 msgr2=0x7fdec4160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec2ffd640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec40b5230 0x7fdec4160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec2ffd640 1 -- 172.21.15.94:0/2607989109 >> v1:172.21.15.138:6789/0 conn(0x7fdec4164b60 legacy=0x7fdec40d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec2ffd640 1 -- 172.21.15.94:0/2607989109 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec40d58f0 con 0x7fdec4164210 2024-04-25T16:27:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.022+0000 7fdec37fe640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec40b5230 0x7fdec4160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.023+0000 7fdec2ffd640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec4164210 0x7fdec4164620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80097a0 tx=0x7fdeb802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2bc23e050c1b975 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.023+0000 7fdec27fc640 1 -- 172.21.15.94:0/2607989109 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8037070 con 0x7fdec4164210 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.023+0000 7fdec27fc640 1 -- 172.21.15.94:0/2607989109 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb802ae50 con 0x7fdec4164210 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.023+0000 7fdec27fc640 1 -- 172.21.15.94:0/2607989109 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8032660 con 0x7fdec4164210 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 -- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec4164210 msgr2=0x7fdec4164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec4164210 0x7fdec4164620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80097a0 tx=0x7fdeb802a100 comp rx=0 tx=0).stop 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 -- 172.21.15.94:0/2607989109 shutdown_connections 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec4164210 0x7fdec4164620 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 --2- 172.21.15.94:0/2607989109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec40b5230 0x7fdec4160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 -- 172.21.15.94:0/2607989109 >> 172.21.15.94:0/2607989109 conn(0x7fdec40d3570 msgr2=0x7fdec40b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 -- 172.21.15.94:0/2607989109 shutdown_connections 2024-04-25T16:27:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.024+0000 7fdeca00c640 1 -- 172.21.15.94:0/2607989109 wait complete. 2024-04-25T16:27:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.026+0000 7fdeca00c640 1 Processor -- start 2024-04-25T16:27:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.026+0000 7fdeca00c640 1 -- start start 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdeca00c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec40b5230 0x7fdec415db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdeca00c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 0x7fdec41fb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdeca00c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec4164b60 0x7fdec40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdeca00c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4164f70 con 0x7fdec4164210 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdeca00c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdec406deb0 con 0x7fdec40b5230 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdeca00c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdec40b96f0 con 0x7fdec4164b60 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdec2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 0x7fdec41fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdec2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 0x7fdec41fb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59106/0 (socket says 172.21.15.94:59106) 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdec2ffd640 1 -- 172.21.15.94:0/4170928759 learned_addr learned my addr 172.21.15.94:0/4170928759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:32.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdec37fe640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec40b5230 0x7fdec415db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.027+0000 7fdec3fff640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec4164b60 0x7fdec40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdec2ffd640 1 -- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec4164b60 msgr2=0x7fdec40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdec2ffd640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec4164b60 0x7fdec40cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdec2ffd640 1 -- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec40b5230 msgr2=0x7fdec415db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdec2ffd640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec40b5230 0x7fdec415db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdec2ffd640 1 -- 172.21.15.94:0/4170928759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec41fba40 con 0x7fdec4164210 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdec2ffd640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 0x7fdec41fb500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8035040 tx=0x7fdeb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8041020 con 0x7fdec4164210 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb802ac50 con 0x7fdec4164210 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec40cf740 con 0x7fdec4164210 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8037040 con 0x7fdec4164210 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.028+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec40ce440 con 0x7fdec4164210 2024-04-25T16:27:32.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.029+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec41ec3b0 con 0x7fdec4164210 2024-04-25T16:27:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.030+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdeb8044020 con 0x7fdec4164210 2024-04-25T16:27:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.030+0000 7fdeabfff640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb00744e0 0x7fdeb00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.030+0000 7fdec37fe640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb00744e0 0x7fdeb00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.030+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdeb80b5920 con 0x7fdec4164210 2024-04-25T16:27:32.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.031+0000 7fdec37fe640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb00744e0 0x7fdeb00769f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009560 tx=0x7fdeb4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:32.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.035+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdeb804b030 con 0x7fdec4164210 2024-04-25T16:27:32.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.210+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdec406a180 con 0x7fdec4164210 2024-04-25T16:27:32.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.212+0000 7fdeabfff640 1 -- 172.21.15.94:0/4170928759 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18478 (secure 0 0 0) 0x7fdec40cf740 con 0x7fdec4164210 2024-04-25T16:27:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:32.213 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:00.941153+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb00744e0 msgr2=0x7fdeb00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb00744e0 0x7fdeb00769f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009560 tx=0x7fdeb4009190 comp rx=0 tx=0).stop 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 msgr2=0x7fdec41fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 0x7fdec41fb500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8035040 tx=0x7fdeb80046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 shutdown_connections 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdeb00744e0 0x7fdeb00769f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdec4164b60 0x7fdec40cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec4164210 0x7fdec41fb500 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 --2- 172.21.15.94:0/4170928759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdec40b5230 0x7fdec415db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 >> 172.21.15.94:0/4170928759 conn(0x7fdec40d3570 msgr2=0x7fdec415e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:32.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 shutdown_connections 2024-04-25T16:27:32.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.214+0000 7fdeca00c640 1 -- 172.21.15.94:0/4170928759 wait complete. 2024-04-25T16:27:32.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:27:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.334+0000 7f7a03c15640 1 Processor -- start 2024-04-25T16:27:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.334+0000 7f7a03c15640 1 -- start start 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.335+0000 7f7a03c15640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0b5230 0x7f79fc160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a03c15640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79fc164210 0x7f79fc164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a03c15640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc0b5a30 con 0x7f79fc164b60 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a03c15640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc06ca10 con 0x7f79fc164210 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a03c15640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0d5610 con 0x7f79fc0b5230 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a011b0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79fc164210 0x7f79fc164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a021b2640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f79fc164b60 0x7f79fc0d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46678/0 (socket says 172.21.15.94:46678) 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a021b2640 1 -- 172.21.15.94:0/311229020 learned_addr learned my addr 172.21.15.94:0/311229020 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a011b0640 1 --2- 172.21.15.94:0/311229020 >> v2:172.21.15.94:3300/0 conn(0x7f79fc164210 0x7f79fc164620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59120/0 (socket says 172.21.15.94:59120) 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a019b1640 1 --2- 172.21.15.94:0/311229020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0b5230 0x7f79fc160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a011b0640 1 -- 172.21.15.94:0/311229020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0b5230 msgr2=0x7f79fc160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a011b0640 1 --2- 172.21.15.94:0/311229020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0b5230 0x7f79fc160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a011b0640 1 -- 172.21.15.94:0/311229020 >> v1:172.21.15.94:6789/0 conn(0x7f79fc164b60 legacy=0x7f79fc0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a011b0640 1 -- 172.21.15.94:0/311229020 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc0d58f0 con 0x7f79fc164210 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.336+0000 7f7a009af640 1 -- 172.21.15.94:0/311229020 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101086071 0 0) 0x7f79f4001490 con 0x7f79fc164b60 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.337+0000 7f7a011b0640 1 --2- 172.21.15.94:0/311229020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc164210 0x7f79fc164620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0097a0 tx=0x7f79ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e52680b4f3ef23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.337+0000 7f7a009af640 1 -- 172.21.15.94:0/311229020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec037070 con 0x7f79fc164210 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.337+0000 7f7a009af640 1 -- 172.21.15.94:0/311229020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec02ae50 con 0x7f79fc164210 2024-04-25T16:27:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.337+0000 7f7a009af640 1 -- 172.21.15.94:0/311229020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec032660 con 0x7f79fc164210 2024-04-25T16:27:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.337+0000 7f7a03c15640 1 -- 172.21.15.94:0/311229020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc164210 msgr2=0x7f79fc164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.337+0000 7f7a03c15640 1 --2- 172.21.15.94:0/311229020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc164210 0x7f79fc164620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0097a0 tx=0x7f79ec02a100 comp rx=0 tx=0).stop 2024-04-25T16:27:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.338+0000 7f7a03c15640 1 -- 172.21.15.94:0/311229020 shutdown_connections 2024-04-25T16:27:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.338+0000 7f7a03c15640 1 --2- 172.21.15.94:0/311229020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc164210 0x7f79fc164620 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.338+0000 7f7a03c15640 1 --2- 172.21.15.94:0/311229020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0b5230 0x7f79fc160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.338+0000 7f7a03c15640 1 -- 172.21.15.94:0/311229020 >> 172.21.15.94:0/311229020 conn(0x7f79fc0d3570 msgr2=0x7f79fc0b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.338+0000 7f7a03c15640 1 -- 172.21.15.94:0/311229020 shutdown_connections 2024-04-25T16:27:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.338+0000 7f7a03c15640 1 -- 172.21.15.94:0/311229020 wait complete. 2024-04-25T16:27:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.339+0000 7f7a03c15640 1 Processor -- start 2024-04-25T16:27:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.339+0000 7f7a03c15640 1 -- start start 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 0x7f79fc15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc164210 0x7f79fc0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79fc164b60 0x7f79fc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc164f70 con 0x7f79fc0b5230 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc06deb0 con 0x7f79fc164b60 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0b96f0 con 0x7f79fc164210 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 0x7f79fc15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 0x7f79fc15b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59134/0 (socket says 172.21.15.94:59134) 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 -- 172.21.15.94:0/2529604999 learned_addr learned my addr 172.21.15.94:0/2529604999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a011b0640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc164210 0x7f79fc0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a021b2640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79fc164b60 0x7f79fc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 -- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc164210 msgr2=0x7f79fc0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc164210 0x7f79fc0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 -- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79fc164b60 msgr2=0x7f79fc15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79fc164b60 0x7f79fc15e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 -- 172.21.15.94:0/2529604999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc0cae30 con 0x7f79fc0b5230 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a019b1640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 0x7f79fc15b620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79f800d5f0 tx=0x7f79f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f80125f0 con 0x7f79fc0b5230 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f8019070 con 0x7f79fc0b5230 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc0cb110 con 0x7f79fc0b5230 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f80129d0 con 0x7f79fc0b5230 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.341+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc0cb6d0 con 0x7f79fc0b5230 2024-04-25T16:27:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.343+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc06a180 con 0x7f79fc0b5230 2024-04-25T16:27:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.343+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79f8012b70 con 0x7f79fc0b5230 2024-04-25T16:27:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.343+0000 7f79f27fc640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f79e00744b0 0x7f79e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.343+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f79f8095a00 con 0x7f79fc0b5230 2024-04-25T16:27:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.343+0000 7f7a011b0640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f79e00744b0 0x7f79e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.344+0000 7f7a011b0640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f79e00744b0 0x7f79e00769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79ec034000 tx=0x7f79ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.348+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79f805fe50 con 0x7f79fc0b5230 2024-04-25T16:27:32.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.561+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79fc0cab90 con 0x7f79fc0b5230 2024-04-25T16:27:32.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.562+0000 7f79f27fc640 1 -- 172.21.15.94:0/2529604999 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f79fc0cab90 con 0x7f79fc0b5230 2024-04-25T16:27:32.563 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:18.975101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:27:32.563 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f79e00744b0 msgr2=0x7f79e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f79e00744b0 0x7f79e00769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79ec034000 tx=0x7f79ec02e000 comp rx=0 tx=0).stop 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 msgr2=0x7f79fc15b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 0x7f79fc15b620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79f800d5f0 tx=0x7f79f800dab0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 shutdown_connections 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f79e00744b0 0x7f79e00769c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79fc164b60 0x7f79fc15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc164210 0x7f79fc0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 --2- 172.21.15.94:0/2529604999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79fc0b5230 0x7f79fc15b620 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 >> 172.21.15.94:0/2529604999 conn(0x7f79fc0d3570 msgr2=0x7f79fc0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:32.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 shutdown_connections 2024-04-25T16:27:32.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:32.564+0000 7f7a03c15640 1 -- 172.21.15.94:0/2529604999 wait complete. 2024-04-25T16:27:32.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T16:26:55.900210+0000', 'modified': '2024-04-25T16:27:18.975101+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6834/4275391000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 4275391000}, {'type': 'v1', 'addr': '172.21.15.138:6834', 'nonce': 4275391000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-25T16:27:32.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T16:27:32.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T16:27:33.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.087+0000 7f0a04f9d640 1 Processor -- start 2024-04-25T16:27:33.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.087+0000 7f0a04f9d640 1 -- start start 2024-04-25T16:27:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f0a04f9d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a0006dc40 0x7f0a0006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f0a04f9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a00160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f0a04f9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a000d0ec0 0x7f0a000d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f0a04f9d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0006d7b0 con 0x7f0a000d0ec0 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f0a04f9d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a000d1810 con 0x7f0a0006dc40 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f0a04f9d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0006cba0 con 0x7f0a00164220 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a000d0ec0 0x7f0a000d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a000d0ec0 0x7f0a000d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59144/0 (socket says 172.21.15.94:59144) 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 -- 172.21.15.94:0/2941324829 learned_addr learned my addr 172.21.15.94:0/2941324829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/2941324829 >> v2:172.21.15.138:3300/0 conn(0x7f0a0006dc40 0x7f0a0006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09feffd640 1 --2- 172.21.15.94:0/2941324829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a00160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 -- 172.21.15.94:0/2941324829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 msgr2=0x7f0a00160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 --2- 172.21.15.94:0/2941324829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a00160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 -- 172.21.15.94:0/2941324829 >> v2:172.21.15.138:3300/0 conn(0x7f0a0006dc40 msgr2=0x7f0a0006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 --2- 172.21.15.94:0/2941324829 >> v2:172.21.15.138:3300/0 conn(0x7f0a0006dc40 0x7f0a0006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.089+0000 7f09fffff640 1 -- 172.21.15.94:0/2941324829 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0006ce80 con 0x7f0a000d0ec0 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.090+0000 7f09fffff640 1 --2- 172.21.15.94:0/2941324829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000d0ec0 0x7f0a000d12d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0a0008ece0 tx=0x7f09f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25af55239d423c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.090+0000 7f09fe7fc640 1 -- 172.21.15.94:0/2941324829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8036070 con 0x7f0a000d0ec0 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.090+0000 7f09fe7fc640 1 -- 172.21.15.94:0/2941324829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f8031600 con 0x7f0a000d0ec0 2024-04-25T16:27:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.090+0000 7f09fe7fc640 1 -- 172.21.15.94:0/2941324829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80319d0 con 0x7f0a000d0ec0 2024-04-25T16:27:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 -- 172.21.15.94:0/2941324829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000d0ec0 msgr2=0x7f0a000d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/2941324829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000d0ec0 0x7f0a000d12d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0a0008ece0 tx=0x7f09f802a6a0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 -- 172.21.15.94:0/2941324829 shutdown_connections 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/2941324829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000d0ec0 0x7f0a000d12d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/2941324829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a00160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/2941324829 >> v2:172.21.15.138:3300/0 conn(0x7f0a0006dc40 0x7f0a0006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 -- 172.21.15.94:0/2941324829 >> 172.21.15.94:0/2941324829 conn(0x7f0a000d6080 msgr2=0x7f0a000d6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 -- 172.21.15.94:0/2941324829 shutdown_connections 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.091+0000 7f0a04f9d640 1 -- 172.21.15.94:0/2941324829 wait complete. 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.093+0000 7f0a04f9d640 1 Processor -- start 2024-04-25T16:27:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.093+0000 7f0a04f9d640 1 -- start start 2024-04-25T16:27:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f0a04f9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 0x7f0a00163380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f0a04f9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a000d0ec0 0x7f0a001faa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f0a04f9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a000ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f0a04f9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00160ac0 con 0x7f0a0006dc40 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f0a04f9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0006d440 con 0x7f0a000d0ec0 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f0a04f9d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a000d1b40 con 0x7f0a00164220 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f09ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 0x7f0a00163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f09ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 0x7f0a00163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59150/0 (socket says 172.21.15.94:59150) 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1686069883 learned_addr learned my addr 172.21.15.94:0/1686069883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.094+0000 7f09feffd640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a000d0ec0 0x7f0a001faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 msgr2=0x7f0a000ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a000ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a000d0ec0 msgr2=0x7f0a001faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a000d0ec0 0x7f0a001faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1686069883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a001611c0 con 0x7f0a0006dc40 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09feffd640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a000d0ec0 0x7f0a001faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 0x7f0a00163380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09f000d800 tx=0x7f09f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09e7fff640 1 -- 172.21.15.94:0/1686069883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0018030 con 0x7f0a0006dc40 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09e7fff640 1 -- 172.21.15.94:0/1686069883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f0009640 con 0x7f0a0006dc40 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a001614a0 con 0x7f0a0006dc40 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f09e7fff640 1 -- 172.21.15.94:0/1686069883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0009810 con 0x7f0a0006dc40 2024-04-25T16:27:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.095+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a00161a60 con 0x7f0a0006dc40 2024-04-25T16:27:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.097+0000 7f09e7fff640 1 -- 172.21.15.94:0/1686069883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09f001b070 con 0x7f0a0006dc40 2024-04-25T16:27:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.097+0000 7f09e7fff640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f09ec0744b0 0x7f09ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.097+0000 7f09e7fff640 1 -- 172.21.15.94:0/1686069883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09f00956c0 con 0x7f0a0006dc40 2024-04-25T16:27:33.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.097+0000 7f09feffd640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f09ec0744b0 0x7f09ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.097+0000 7f09feffd640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f09ec0744b0 0x7f09ec0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09f4006fd0 tx=0x7f09f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.098+0000 7f0a0d18f740 1 Processor -- start 2024-04-25T16:27:33.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.098+0000 7f0a0d18f740 1 -- start start 2024-04-25T16:27:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.099+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e53270 0x55703521f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.099+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x55703521c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.099+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x5570351f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557035159740 con 0x557034e2a3b0 2024-04-25T16:27:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570350f53a0 con 0x557034e32590 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557035155700 con 0x557034e53270 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x55703521c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x55703521c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59156/0 (socket says 172.21.15.94:59156) 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 -- 172.21.15.94:0/2728511763 learned_addr learned my addr 172.21.15.94:0/2728511763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e53270 0x55703521f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 -- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e53270 msgr2=0x55703521f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09fffff640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x5570351f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e53270 0x55703521f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 -- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 msgr2=0x5570351f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x5570351f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 -- 172.21.15.94:0/2728511763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557035111750 con 0x557034e2a3b0 2024-04-25T16:27:33.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e53270 0x55703521f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09fffff640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x5570351f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09feffd640 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x55703521c1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09f4004380 tx=0x7f09f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec7de92acacf750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09e57fa640 1 -- 172.21.15.94:0/2728511763 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f400f040 con 0x557034e2a3b0 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09e57fa640 1 -- 172.21.15.94:0/2728511763 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f40043b0 con 0x557034e2a3b0 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.100+0000 7f09e57fa640 1 -- 172.21.15.94:0/2728511763 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f400ee80 con 0x557034e2a3b0 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 -- 172.21.15.94:0/2728511763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 msgr2=0x55703521c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x55703521c1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09f4004380 tx=0x7f09f4011000 comp rx=0 tx=0).stop 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 -- 172.21.15.94:0/2728511763 shutdown_connections 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x5570351f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x55703521c1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/2728511763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e53270 0x55703521f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 -- 172.21.15.94:0/2728511763 >> 172.21.15.94:0/2728511763 conn(0x557034e473c0 msgr2=0x5570351f4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 -- 172.21.15.94:0/2728511763 shutdown_connections 2024-04-25T16:27:33.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.101+0000 7f0a0d18f740 1 -- 172.21.15.94:0/2728511763 wait complete. 2024-04-25T16:27:33.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.102+0000 7f0a0d18f740 1 -- start start 2024-04-25T16:27:33.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.102+0000 7f0a0d18f740 1 Processor -- start 2024-04-25T16:27:33.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035222560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x55703523a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 0x55703523aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570350e0fe0 con 0x557034e53270 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557035204630 con 0x557034e32590 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55703521b300 con 0x557034e2a3b0 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.103+0000 7f0a0d18f740 10 client.0 _ll_register_callbacks cb 0x5570350f7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 0x55703523aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 0x55703523aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59164/0 (socket says 172.21.15.94:59164) 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 -- 172.21.15.94:0/1133306964 learned_addr learned my addr 172.21.15.94:0/1133306964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035222560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09feffd640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x55703523a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 -- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 msgr2=0x557035222560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035222560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 -- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 msgr2=0x55703523a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x55703523a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 -- 172.21.15.94:0/1133306964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557034e6b2d0 con 0x557034e53270 2024-04-25T16:27:33.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09fffff640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 0x55703523aac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09f802a6e0 tx=0x7f09f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526aef658b6b7efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09de7fc640 1 -- 172.21.15.94:0/1133306964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8040020 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f40043b0 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09de7fc640 1 -- 172.21.15.94:0/1133306964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09f8031be0 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09de7fc640 1 -- 172.21.15.94:0/1133306964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f09f804a050 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f0a0d18f740 10 client.4809 fetch_fsmap learned FSMap version 9 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x557034f356b0 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.104+0000 7f09de7fc640 1 -- 172.21.15.94:0/1133306964 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f40043b0 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f09de7fc640 1 -- 172.21.15.94:0/1133306964 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f09f804d020 con 0x557034e53270 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f0a0d18f740 10 client.4809 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f0a0d18f740 10 client.4809 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f0a0d18f740 20 client.4809 populate_metadata read hostname 'smithi094' 2024-04-25T16:27:33.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034e11e40 0x5570352036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f0a0d18f740 4 client.4809 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 --> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557035167350 con 0x557034e11e40 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034e11e40 0x5570352036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034e11e40 0x5570352036d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9f3b040b3dd09f server_cookie=e89195213968bf3b in_seq=0 out_seq=0 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.105+0000 7f09de7fc640 10 client.4809 ms_handle_connect on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f09de7fc640 1 -- 172.21.15.94:0/1133306964 <== mds.0 v2:172.21.15.138:6832/4275391000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557035167350 con 0x557034e11e40 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f09de7fc640 10 client.4809 handle_command_reply: tid=0 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 client.4809 shutdown 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 msgr2=0x55703523aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 0x55703523aac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09f802a6e0 tx=0x7f09f8004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 shutdown_connections 2024-04-25T16:27:33.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034e11e40 0x5570352036d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e53270 0x55703523aac0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x55703523a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f09de7fc640 0 client.4809 ms_handle_reset on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1133306964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035222560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 >> 172.21.15.94:0/1133306964 conn(0x557034e473c0 msgr2=0x5570351f3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 shutdown_connections 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1133306964 wait complete. 2024-04-25T16:27:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.106+0000 7f0a0d18f740 20 client.4809 trim_cache size 0 max 16384 2024-04-25T16:27:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.125+0000 7f0a0d18f740 1 Processor -- start 2024-04-25T16:27:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.125+0000 7f0a0d18f740 1 -- start start 2024-04-25T16:27:33.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e473c0 0x557035222e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035223340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 0x5570351f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00203520 con 0x557034e32590 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001f7880 con 0x557034e473c0 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a00202ed0 con 0x557034e2a3b0 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f09fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 0x5570351f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f09fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 0x5570351f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59176/0 (socket says 172.21.15.94:59176) 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.126+0000 7f09fffff640 1 -- 172.21.15.94:0/3986214413 learned_addr learned my addr 172.21.15.94:0/3986214413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09feffd640 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035223340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e473c0 0x557035222e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09fffff640 1 -- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 msgr2=0x557035223340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09fffff640 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035223340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09fffff640 1 -- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e473c0 msgr2=0x557035222e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09fffff640 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e473c0 0x557035222e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09fffff640 1 -- 172.21.15.94:0/3986214413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703523b0d0 con 0x557034e32590 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09fffff640 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 0x5570351f5be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09f8007100 tx=0x7f09f8037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbea674684d9197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09e4ff9640 1 -- 172.21.15.94:0/3986214413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8040070 con 0x557034e32590 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09e4ff9640 1 -- 172.21.15.94:0/3986214413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f8009510 con 0x557034e32590 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.127+0000 7f09e4ff9640 1 -- 172.21.15.94:0/3986214413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8036040 con 0x557034e32590 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 -- 172.21.15.94:0/3986214413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 msgr2=0x5570351f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 0x5570351f5be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09f8007100 tx=0x7f09f8037000 comp rx=0 tx=0).stop 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 -- 172.21.15.94:0/3986214413 shutdown_connections 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e32590 0x5570351f5be0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e2a3b0 0x557035223340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/3986214413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e473c0 0x557035222e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 -- 172.21.15.94:0/3986214413 >> 172.21.15.94:0/3986214413 conn(0x557035220aa0 msgr2=0x55703523c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 -- 172.21.15.94:0/3986214413 shutdown_connections 2024-04-25T16:27:33.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.128+0000 7f0a0d18f740 1 -- 172.21.15.94:0/3986214413 wait complete. 2024-04-25T16:27:33.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.129+0000 7f0a0d18f740 1 -- start start 2024-04-25T16:27:33.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.129+0000 7f0a0d18f740 1 Processor -- start 2024-04-25T16:27:33.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x557035247770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x557035247cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e473c0 0x557035218b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557035222c80 con 0x557034e2a3b0 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557035205100 con 0x557034e32590 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55703521b0d0 con 0x557034e473c0 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f0a0d18f740 10 client.0 _ll_register_callbacks cb 0x7f09f8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x557035247770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x557035247770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59178/0 (socket says 172.21.15.94:59178) 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1827356030 learned_addr learned my addr 172.21.15.94:0/1827356030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09feffd640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x557035247cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09fffff640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e473c0 0x557035218b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e473c0 msgr2=0x557035218b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e473c0 0x557035218b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 msgr2=0x557035247cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x557035247cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09ff7fe640 1 -- 172.21.15.94:0/1827356030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557035223710 con 0x557034e2a3b0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.130+0000 7f09fffff640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e473c0 0x557035218b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09ff7fe640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x557035247770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09f005ee10 tx=0x7f09f000b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96be99411bbb04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09de7fc640 1 -- 172.21.15.94:0/1827356030 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0066dd0 con 0x557034e2a3b0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09de7fc640 1 -- 172.21.15.94:0/1827356030 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09f0067e70 con 0x557034e2a3b0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f8009510 con 0x557034e2a3b0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09de7fc640 1 -- 172.21.15.94:0/1827356030 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f09f0068070 con 0x557034e2a3b0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 10 client.4815 fetch_fsmap learned FSMap version 9 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x557035230c30 con 0x557034e2a3b0 2024-04-25T16:27:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09de7fc640 1 -- 172.21.15.94:0/1827356030 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f8009510 con 0x557034e2a3b0 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09de7fc640 1 -- 172.21.15.94:0/1827356030 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f09f0025060 con 0x557034e2a3b0 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 10 client.4815 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 10 client.4815 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 20 client.4815 populate_metadata read hostname 'smithi094' 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034de7280 0x5570352191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 4 client.4815 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 --> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] -- command(tid 0: {"prefix": "status"}) -- 0x5570351f0160 con 0x557034de7280 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.131+0000 7f09feffd640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034de7280 0x5570352191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f09de7fc640 10 client.4815 ms_handle_connect on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f09feffd640 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034de7280 0x5570352191e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ef9273033395a2 server_cookie=b1ddb5ac5e108d00 in_seq=0 out_seq=0 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f09de7fc640 1 -- 172.21.15.94:0/1827356030 <== mds.0 v2:172.21.15.138:6832/4275391000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5570351f0160 con 0x557034de7280 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f09de7fc640 10 client.4815 handle_command_reply: tid=0 2024-04-25T16:27:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 client.4815 shutdown 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 msgr2=0x557035247770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x557035247770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09f005ee10 tx=0x7f09f000b450 comp rx=0 tx=0).stop 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 shutdown_connections 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557034e473c0 0x557035218b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557034e32590 0x557035247cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.138:6832/4275391000,v1:172.21.15.138:6834/4275391000] conn(0x557034de7280 0x5570352191e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 --2- 172.21.15.94:0/1827356030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557034e2a3b0 0x557035247770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f09de7fc640 0 client.4815 ms_handle_reset on v2:172.21.15.138:6832/4275391000 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 >> 172.21.15.94:0/1827356030 conn(0x557035220aa0 msgr2=0x557035220f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 shutdown_connections 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 1 -- 172.21.15.94:0/1827356030 wait complete. 2024-04-25T16:27:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.132+0000 7f0a0d18f740 20 client.4815 trim_cache size 0 max 16384 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "e3c627c0-5913-4b5c-b1aa-d445103b794b", 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4279, 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 36.596192281, 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 9, 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 25, 2024-04-25T16:27:33.140 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T16:27:33.141 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 44.070132635999997 2024-04-25T16:27:33.141 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:27:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f09ec0744b0 msgr2=0x7f09ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f09ec0744b0 0x7f09ec0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09f4006fd0 tx=0x7f09f4008040 comp rx=0 tx=0).stop 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 msgr2=0x7f0a00163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 0x7f0a00163380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09f000d800 tx=0x7f09f000dcc0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 shutdown_connections 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f09ec0744b0 0x7f09ec0769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a00164220 0x7f0a000ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a000d0ec0 0x7f0a001faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 --2- 172.21.15.94:0/1686069883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0006dc40 0x7f0a00163380 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 >> 172.21.15.94:0/1686069883 conn(0x7f0a000d6080 msgr2=0x7f0a001e9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 shutdown_connections 2024-04-25T16:27:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.133+0000 7f0a04f9d640 1 -- 172.21.15.94:0/1686069883 wait complete. 2024-04-25T16:27:33.145 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T16:27:33.155 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T16:27:33.155 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T16:27:33.155 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T16:27:33.155 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T16:27:33.167 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T16:27:33.167 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T16:27:33.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.167 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:33.182 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T16:27:33.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:27:33.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T16:27:33.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.331 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:33.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.345 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T16:27:33.411 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-04-25T16:27:33.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:27:33.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.413 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:27:33.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T16:27:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-04-25T16:27:33.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:27:33.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.504 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:27:33.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.561 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T16:27:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-04-25T16:27:33.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:27:33.630 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T16:27:33.630 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:27:33.630 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:27:33.630 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.737+0000 7f9081bce640 1 Processor -- start 2024-04-25T16:27:33.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.737+0000 7f9081bce640 1 -- start start 2024-04-25T16:27:33.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f9081bce640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f907c06dc90 0x7f907c0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f9081bce640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0b1650 con 0x7f907c15d3e0 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f9081bce640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c160fd0 con 0x7f907c161ab0 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f9081bce640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f907c0b1410 con 0x7f907c06dc90 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f907b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f907c15d3e0 0x7f907c0b0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46694/0 (socket says 172.21.15.94:46694) 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f907b7fe640 1 -- 172.21.15.94:0/2487146920 learned_addr learned my addr 172.21.15.94:0/2487146920 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f907bfff640 1 --2- 172.21.15.94:0/2487146920 >> v2:172.21.15.138:3300/0 conn(0x7f907c06dc90 0x7f907c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91666271 0 0) 0x7f907c0b1650 con 0x7f907c15d3e0 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.738+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9064003520 con 0x7f907c15d3e0 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3723739738 0 0) 0x7f9064003520 con 0x7f907c15d3e0 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 >> v2:172.21.15.138:3300/0 conn(0x7f907c06dc90 msgr2=0x7f907c0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 --2- 172.21.15.94:0/2487146920 >> v2:172.21.15.138:3300/0 conn(0x7f907c06dc90 0x7f907c0b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 >> v1:172.21.15.138:6789/0 conn(0x7f907c161ab0 legacy=0x7f907c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c1612b0 con 0x7f907c15d3e0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f906c001670 con 0x7f907c15d3e0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 >> v1:172.21.15.94:6789/0 conn(0x7f907c15d3e0 legacy=0x7f907c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90640039b0 0x7f9064003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9064004800 0x7f9064004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90640051c0 0x7f906400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0b1410 con 0x7f90640039b0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f906400c160 con 0x7f90640051c0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f906400c360 con 0x7f9064004800 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907b7fe640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90640039b0 0x7f9064003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907bfff640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90640051c0 0x7f906400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907affd640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9064004800 0x7f9064004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907b7fe640 1 -- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9064004800 msgr2=0x7f9064004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907b7fe640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9064004800 0x7f9064004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907b7fe640 1 -- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90640051c0 msgr2=0x7f906400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907b7fe640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90640051c0 0x7f906400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907b7fe640 1 -- 172.21.15.94:0/2487146920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9064003520 con 0x7f90640039b0 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907bfff640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90640051c0 0x7f906400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.739+0000 7f907bfff640 1 -- 172.21.15.94:0/2487146920 reap_dead start 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.740+0000 7f907b7fe640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90640039b0 0x7f9064003e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c210 tx=0x7f9064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb5e95fed046589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.740+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c007d30 con 0x7f90640039b0 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.740+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c005af0 con 0x7f90640039b0 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.740+0000 7f907a7fc640 1 -- 172.21.15.94:0/2487146920 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c007070 con 0x7f90640039b0 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 -- 172.21.15.94:0/2487146920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90640039b0 msgr2=0x7f9064003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90640039b0 0x7f9064003e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c210 tx=0x7f9064005830 comp rx=0 tx=0).stop 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 -- 172.21.15.94:0/2487146920 shutdown_connections 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 --2- 172.21.15.94:0/2487146920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90640039b0 0x7f9064003e20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 -- 172.21.15.94:0/2487146920 >> 172.21.15.94:0/2487146920 conn(0x7f907c0d23b0 msgr2=0x7f907c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 -- 172.21.15.94:0/2487146920 shutdown_connections 2024-04-25T16:27:33.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.741+0000 7f9081bce640 1 -- 172.21.15.94:0/2487146920 wait complete. 2024-04-25T16:27:33.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.742+0000 7f9081bce640 1 Processor -- start 2024-04-25T16:27:33.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.743+0000 7f9081bce640 1 -- start start 2024-04-25T16:27:33.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f9081bce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90640039b0 0x7f907c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f9081bce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 0x7f907c151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f9081bce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f907c15c1d0 0x7f907c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f9081bce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f907c161620 con 0x7f907c15d8b0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f9081bce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0b6b80 con 0x7f907c15c1d0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f9081bce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f907c0b0d00 con 0x7f90640039b0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 0x7f907c151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 0x7f907c151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59192/0 (socket says 172.21.15.94:59192) 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 -- 172.21.15.94:0/1071389689 learned_addr learned my addr 172.21.15.94:0/1071389689 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907b7fe640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90640039b0 0x7f907c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907bfff640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f907c15c1d0 0x7f907c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 -- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90640039b0 msgr2=0x7f907c152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90640039b0 0x7f907c152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 -- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f907c15c1d0 msgr2=0x7f907c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f907c15c1d0 0x7f907c15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 -- 172.21.15.94:0/1071389689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c15cb20 con 0x7f907c15d8b0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.744+0000 7f907affd640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 0x7f907c151180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f907000e4d0 tx=0x7f907000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.745+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90700053a0 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.745+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9070018070 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.745+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c160160 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.745+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070003810 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.745+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c15ddd0 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.746+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c15f370 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.746+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f90700039b0 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.747+0000 7f9063fff640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f90680744e0 0x7f90680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.747+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f90700951a0 con 0x7f907c15d8b0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.747+0000 7f907b7fe640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f90680744e0 0x7f90680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.747+0000 7f907b7fe640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f90680744e0 0x7f90680769f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f906c032000 tx=0x7f906c02d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.751+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f907005f710 con 0x7f907c15d8b0 2024-04-25T16:27:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.963+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f907c0b3280 con 0x7f907c15d8b0 2024-04-25T16:27:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.963+0000 7f9063fff640 1 -- 172.21.15.94:0/1071389689 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f907c0b3280 con 0x7f907c15d8b0 2024-04-25T16:27:33.964 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:27:33.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f90680744e0 msgr2=0x7f90680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f90680744e0 0x7f90680769f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f906c032000 tx=0x7f906c02d000 comp rx=0 tx=0).stop 2024-04-25T16:27:33.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 msgr2=0x7f907c151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:33.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 0x7f907c151180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f907000e4d0 tx=0x7f907000e990 comp rx=0 tx=0).stop 2024-04-25T16:27:33.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 shutdown_connections 2024-04-25T16:27:33.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f90680744e0 0x7f90680769f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f907c15c1d0 0x7f907c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f907c15d8b0 0x7f907c151180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 --2- 172.21.15.94:0/1071389689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90640039b0 0x7f907c152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 >> 172.21.15.94:0/1071389689 conn(0x7f907c0d23b0 msgr2=0x7f907c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.965+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 shutdown_connections 2024-04-25T16:27:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:33.966+0000 7f9081bce640 1 -- 172.21.15.94:0/1071389689 wait complete. 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:27:33.977 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:27:33.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:33.978 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:33.994 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:27:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T16:27:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42679sec preferred_lft 42679sec 2024-04-25T16:27:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::55c9:4b97:ccb7:f8e8/64 scope link noprefixroute 2024-04-25T16:27:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:33.994 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:27:33.994 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:33.994 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:33.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:27:33.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-25T16:27:33.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-25T16:27:33.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:27:33.995 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:34.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:34.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:34.127 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:27:34.148 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:27:34.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:34.150 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:27:34.206 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:27:34.207 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:27:34.207 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:27:34.207 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:27:34.207 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:27:34.207 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:34.207 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:34.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:27:34.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:27:34.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:27:34.207 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:34.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:34.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:34.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:34.331 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:34.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:34.345 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T16:27:34.399 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:34.399 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:34.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:27:34.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T16:27:34.400 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:34.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:34.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:34.514 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T16:27:34.515 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:34.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:34.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:34.622 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:34.622 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:34.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T16:27:34.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:27:34.622 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:34.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:34.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:34.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:34.697 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T16:27:34.711 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:27:34.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:34.711 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:27:34.765 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T16:27:34.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:34.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T16:27:34.857 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T16:27:34.857 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T16:27:34.857 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T16:27:34.965 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T16:27:34.965 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T16:27:34.965 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T16:27:34.965 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:34.965 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T16:27:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T16:27:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T16:27:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e3c627c0-5913-4b5c-b1aa-d445103b794b" will pass to kernel 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T16:27:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T16:27:34.974 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T16:27:34.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:34.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:27:35.003 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:35.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:35.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:client4726 2024-04-25T16:27:35.099 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:35.099 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:35.114 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:35.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:35.210 INFO:teuthology.orchestra.run.smithi094.stdout:client4726 2024-04-25T16:27:35.210 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:35.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:35.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:35.265 INFO:teuthology.orchestra.run.smithi094.stdout:client4726 2024-04-25T16:27:35.265 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e3c627c0-5913-4b5c-b1aa-d445103b794b.client4726/status') 2024-04-25T16:27:35.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:35.323 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.4726 (3)172.21.15.94:0/2525036988 2024-04-25T16:27:35.323 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-04-25T16:27:35.324 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-04-25T16:27:35.324 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-04-25T16:27:35.324 INFO:teuthology.orchestra.run.smithi094.stderr:70 bytes copied, 2.937e-05 s, 2.4 MB/s 2024-04-25T16:27:35.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:35.331 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T16:27:35.331 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:27:35.331 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T16:27:35.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:27:35.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.439+0000 7f2d9d584640 1 Processor -- start 2024-04-25T16:27:35.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.439+0000 7f2d9d584640 1 -- start start 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.440+0000 7f2d9d584640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d9806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.440+0000 7f2d9d584640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9806c9c0 con 0x7f2d980d1010 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.440+0000 7f2d9d584640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9806cca0 con 0x7f2d980d4bb0 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.440+0000 7f2d9d584640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9806dfa0 con 0x7f2d9808e780 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d977fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2d980d1010 0x7f2d980d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46708/0 (socket says 172.21.15.94:46708) 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d977fe640 1 -- 172.21.15.94:0/4157824703 learned_addr learned my addr 172.21.15.94:0/4157824703 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460982390 0 0) 0x7f2d9806c9c0 con 0x7f2d980d1010 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d80003520 con 0x7f2d980d1010 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d96ffd640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d9806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 297515159 0 0) 0x7f2d80003520 con 0x7f2d980d1010 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 msgr2=0x7f2d9806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d9806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 >> v1:172.21.15.138:6789/0 conn(0x7f2d980d4bb0 legacy=0x7f2d980b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9806cf80 con 0x7f2d980d1010 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d96ffd640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d9806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f2d900019f0 con 0x7f2d980d1010 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 >> v1:172.21.15.94:6789/0 conn(0x7f2d980d1010 legacy=0x7f2d980d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d80004d40 0x7f2d80005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9806dfa0 con 0x7f2d80003ff0 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8000bbd0 con 0x7f2d80004d40 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8000bdd0 con 0x7f2d9808e780 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d967fc640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d96ffd640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.441+0000 7f2d977fe640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d80004d40 0x7f2d80005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d967fc640 1 -- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 msgr2=0x7f2d800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d967fc640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d9808e780 0x7f2d800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d967fc640 1 -- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d80004d40 msgr2=0x7f2d80005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:35.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d967fc640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d80004d40 0x7f2d80005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d967fc640 1 -- 172.21.15.94:0/4157824703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d80003520 con 0x7f2d80003ff0 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d967fc640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d80004800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c001750 tx=0x7f2d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f8054730688716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c001370 con 0x7f2d80003ff0 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d8c0052d0 con 0x7f2d80003ff0 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.442+0000 7f2d95ffb640 1 -- 172.21.15.94:0/4157824703 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c02abe0 con 0x7f2d80003ff0 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d9d584640 1 -- 172.21.15.94:0/4157824703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 msgr2=0x7f2d80004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d9d584640 1 --2- 172.21.15.94:0/4157824703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d80004800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c001750 tx=0x7f2d80005830 comp rx=0 tx=0).stop 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d977fe640 1 -- 172.21.15.94:0/4157824703 reap_dead start 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d9d584640 1 -- 172.21.15.94:0/4157824703 shutdown_connections 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d9d584640 1 -- 172.21.15.94:0/4157824703 >> 172.21.15.94:0/4157824703 conn(0x7f2d980b7cc0 msgr2=0x7f2d980b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d9d584640 1 -- 172.21.15.94:0/4157824703 shutdown_connections 2024-04-25T16:27:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.443+0000 7f2d9d584640 1 -- 172.21.15.94:0/4157824703 wait complete. 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.445+0000 7f2d9d584640 1 Processor -- start 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.445+0000 7f2d9d584640 1 -- start start 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d9d584640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d981629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d9d584640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d98159a30 0x7f2d98145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d9d584640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d98147670 0x7f2d98147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d9d584640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9806c570 con 0x7f2d80003ff0 2024-04-25T16:27:35.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d9d584640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9806d540 con 0x7f2d98159a30 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d9d584640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9806c6f0 con 0x7f2d98147670 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.446+0000 7f2d96ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d981629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d981629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59200/0 (socket says 172.21.15.94:59200) 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 -- 172.21.15.94:0/539229250 learned_addr learned my addr 172.21.15.94:0/539229250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d967fc640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d98159a30 0x7f2d98145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d977fe640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d98147670 0x7f2d98147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 -- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d98147670 msgr2=0x7f2d98147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d98147670 0x7f2d98147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 -- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d98159a30 msgr2=0x7f2d98145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d98159a30 0x7f2d98145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 -- 172.21.15.94:0/539229250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d98162f00 con 0x7f2d80003ff0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d96ffd640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d981629c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d88011620 tx=0x7f2d88011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88019070 con 0x7f2d80003ff0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d88004220 con 0x7f2d80003ff0 2024-04-25T16:27:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800f040 con 0x7f2d80003ff0 2024-04-25T16:27:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d98147fc0 con 0x7f2d80003ff0 2024-04-25T16:27:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.447+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d98161c00 con 0x7f2d80003ff0 2024-04-25T16:27:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.448+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9806a180 con 0x7f2d80003ff0 2024-04-25T16:27:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.449+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d8800f720 con 0x7f2d80003ff0 2024-04-25T16:27:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.449+0000 7f2d7f7fe640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2d840744e0 0x7f2d840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.449+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2d88095720 con 0x7f2d80003ff0 2024-04-25T16:27:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.449+0000 7f2d967fc640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2d840744e0 0x7f2d840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.450+0000 7f2d967fc640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2d840744e0 0x7f2d840769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c032000 tx=0x7f2d8c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:35.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.454+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d8805fc90 con 0x7f2d80003ff0 2024-04-25T16:27:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.667+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2d980d4640 con 0x7f2d80003ff0 2024-04-25T16:27:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.668+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/539229250 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f2d8805f590 con 0x7f2d80003ff0 2024-04-25T16:27:35.667 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2d840744e0 msgr2=0x7f2d840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2d840744e0 0x7f2d840769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c032000 tx=0x7f2d8c02e000 comp rx=0 tx=0).stop 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 msgr2=0x7f2d981629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d981629c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d88011620 tx=0x7f2d88011ae0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 shutdown_connections 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f2d840744e0 0x7f2d840769f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d98147670 0x7f2d98147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d98159a30 0x7f2d98145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 --2- 172.21.15.94:0/539229250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d80003ff0 0x7f2d981629c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 >> 172.21.15.94:0/539229250 conn(0x7f2d980b7cc0 msgr2=0x7f2d9815f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 shutdown_connections 2024-04-25T16:27:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35.670+0000 7f2d9d584640 1 -- 172.21.15.94:0/539229250 wait complete. 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:27:35.681 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T16:27:35.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:35.681 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:27:35.700 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42678sec preferred_lft 42678sec 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::55c9:4b97:ccb7:f8e8/64 scope link noprefixroute 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 72:9c:2f:b6:54:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::a417:a1ff:fee3:c8b3/64 scope link 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 72:9c:2f:b6:54:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:27:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::709c:2fff:feb6:54c2/64 scope link tentative 2024-04-25T16:27:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:35.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:35.702 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:27:35.763 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:27:35.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:35.764 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:27:35.820 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:27:35.820 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:27:35.820 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:27:35.820 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:27:35.820 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:27:35.820 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:35.820 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:35.820 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:27:35.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:27:35.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:27:35.821 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:35.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:35.943 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:35.957 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T16:27:35.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:35.958 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T16:27:36.012 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T16:27:36.012 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:36.012 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:36.012 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:27:36.012 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T16:27:36.012 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:36.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:36.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:36.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:36.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 3e:f6:63:a2:8b:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:27:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:27:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::3cf6:63ff:fea2:8b70/64 scope link tentative 2024-04-25T16:27:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:36.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 3e:f6:63:a2:8b:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::3cf6:63ff:fea2:8b70/64 scope link tentative 2024-04-25T16:27:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:36.216 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T16:27:36.217 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:36.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:36.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:36.362 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:36.362 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:27:36.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2024-04-25T16:27:36.363 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T16:27:36.363 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:27:36.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:36.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:36.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:36.436 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T16:27:36.451 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T16:27:36.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:36.451 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T16:27:36.505 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T16:27:36.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:36.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T16:27:36.598 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T16:27:36.598 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T16:27:36.598 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T16:27:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T16:27:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T16:27:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T16:27:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-04-25T16:27:36.615 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=e3c627c0-5913-4b5c-b1aa-d445103b794b" will pass to kernel 2024-04-25T16:27:36.616 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T16:27:36.616 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-04-25T16:27:36.616 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T16:27:36.616 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T16:27:36.616 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T16:27:36.617 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T16:27:36.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:36.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T16:27:36.640 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:36.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:37.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:client4735 2024-04-25T16:27:37.301 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:37.301 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:37.318 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:37.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:37.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:37.418 INFO:teuthology.orchestra.run.smithi094.stdout:client4735 2024-04-25T16:27:37.418 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:37.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:37.474 INFO:teuthology.orchestra.run.smithi094.stdout:client4735 2024-04-25T16:27:37.474 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e3c627c0-5913-4b5c-b1aa-d445103b794b.client4735/status') 2024-04-25T16:27:37.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.4735 (3)172.21.15.94:0/2833294797 2024-04-25T16:27:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-04-25T16:27:37.531 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-04-25T16:27:37.531 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-04-25T16:27:37.531 INFO:teuthology.orchestra.run.smithi094.stderr:70 bytes copied, 3.9088e-05 s, 1.8 MB/s 2024-04-25T16:27:37.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:37.538 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T16:27:37.538 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:27:37.538 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T16:27:37.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:27:37.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.646+0000 7f8169172640 1 Processor -- start 2024-04-25T16:27:37.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.646+0000 7f8169172640 1 -- start start 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f8169172640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8164160ab0 0x7f816409c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f8169172640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81640d0420 0x7f81640b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f8169172640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81640b47d0 con 0x7f81640cf900 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f8169172640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f816409c950 con 0x7f8164160ab0 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f8169172640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f81640b4210 con 0x7f81640d0420 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f8162d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8164160ab0 0x7f816409c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f815bfff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f81640cf900 0x7f81640cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46716/0 (socket says 172.21.15.94:46716) 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.647+0000 7f815bfff640 1 -- 172.21.15.94:0/407465567 learned_addr learned my addr 172.21.15.94:0/407465567 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f816359e640 1 --2- 172.21.15.94:0/407465567 >> v2:172.21.15.138:3300/0 conn(0x7f81640d0420 0x7f81640b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:37.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f8162d9d640 1 -- 172.21.15.94:0/407465567 >> v2:172.21.15.138:3300/0 conn(0x7f81640d0420 msgr2=0x7f81640b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f8162d9d640 1 --2- 172.21.15.94:0/407465567 >> v2:172.21.15.138:3300/0 conn(0x7f81640d0420 0x7f81640b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f8162d9d640 1 -- 172.21.15.94:0/407465567 >> v1:172.21.15.94:6789/0 conn(0x7f81640cf900 legacy=0x7f81640cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f8162d9d640 1 -- 172.21.15.94:0/407465567 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640b44f0 con 0x7f8164160ab0 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f816259c640 1 -- 172.21.15.94:0/407465567 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2887298381 0 0) 0x7f81640b47d0 con 0x7f81640cf900 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f8162d9d640 1 --2- 172.21.15.94:0/407465567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f816409c370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f816409c2c0 tx=0x7f815402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e415cb43157145 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.648+0000 7f816259c640 1 -- 172.21.15.94:0/407465567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037070 con 0x7f8164160ab0 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f816259c640 1 -- 172.21.15.94:0/407465567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8154032440 con 0x7f8164160ab0 2024-04-25T16:27:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f816259c640 1 -- 172.21.15.94:0/407465567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815402ae60 con 0x7f8164160ab0 2024-04-25T16:27:37.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 -- 172.21.15.94:0/407465567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 msgr2=0x7f816409c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:37.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 --2- 172.21.15.94:0/407465567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f816409c370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f816409c2c0 tx=0x7f815402a110 comp rx=0 tx=0).stop 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 -- 172.21.15.94:0/407465567 shutdown_connections 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 --2- 172.21.15.94:0/407465567 >> v2:172.21.15.138:3300/0 conn(0x7f81640d0420 0x7f81640b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 --2- 172.21.15.94:0/407465567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f816409c370 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 -- 172.21.15.94:0/407465567 >> 172.21.15.94:0/407465567 conn(0x7f81640b7b90 msgr2=0x7f81640b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 -- 172.21.15.94:0/407465567 shutdown_connections 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.649+0000 7f8169172640 1 -- 172.21.15.94:0/407465567 wait complete. 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.651+0000 7f8169172640 1 Processor -- start 2024-04-25T16:27:37.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.651+0000 7f8169172640 1 -- start start 2024-04-25T16:27:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.652+0000 7f8169172640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81640cf900 0x7f81641503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.652+0000 7f8169172640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d0420 0x7f8164160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.652+0000 7f8169172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f8164163720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.652+0000 7f8169172640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81640b2be0 con 0x7f8164160ab0 2024-04-25T16:27:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.652+0000 7f8169172640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f816406f840 con 0x7f81640d0420 2024-04-25T16:27:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.652+0000 7f8169172640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f81640b2d60 con 0x7f81640cf900 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f8164163720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f8164163720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59216/0 (socket says 172.21.15.94:59216) 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 -- 172.21.15.94:0/3227206144 learned_addr learned my addr 172.21.15.94:0/3227206144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f815bfff640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d0420 0x7f8164160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f8162d9d640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81640cf900 0x7f81641503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 -- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81640cf900 msgr2=0x7f81641503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81640cf900 0x7f81641503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 -- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d0420 msgr2=0x7f8164160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d0420 0x7f8164160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 -- 172.21.15.94:0/3227206144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8164161980 con 0x7f8164160ab0 2024-04-25T16:27:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f816359e640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f8164163720 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f815c007d80 tx=0x7f815c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c014420 con 0x7f8164160ab0 2024-04-25T16:27:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815c019070 con 0x7f8164160ab0 2024-04-25T16:27:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8164161c60 con 0x7f8164160ab0 2024-04-25T16:27:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c014800 con 0x7f8164160ab0 2024-04-25T16:27:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.653+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816415cf40 con 0x7f8164160ab0 2024-04-25T16:27:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.654+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816406a180 con 0x7f8164160ab0 2024-04-25T16:27:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.655+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f815c0149a0 con 0x7f8164160ab0 2024-04-25T16:27:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.655+0000 7f815b7fe640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f81440744e0 0x7f81440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.656+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f815c095db0 con 0x7f8164160ab0 2024-04-25T16:27:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.656+0000 7f8162d9d640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f81440744e0 0x7f81440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.656+0000 7f8162d9d640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f81440744e0 0x7f81440769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8154034000 tx=0x7f815402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.660+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f815c0601d0 con 0x7f8164160ab0 2024-04-25T16:27:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.873+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8164162180 con 0x7f8164160ab0 2024-04-25T16:27:37.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.873+0000 7f815b7fe640 1 -- 172.21.15.94:0/3227206144 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f815c05fad0 con 0x7f8164160ab0 2024-04-25T16:27:37.874 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:27:37.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f81440744e0 msgr2=0x7f81440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:37.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f81440744e0 0x7f81440769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8154034000 tx=0x7f815402e000 comp rx=0 tx=0).stop 2024-04-25T16:27:37.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 msgr2=0x7f8164163720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:37.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f8164163720 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f815c007d80 tx=0x7f815c00e4b0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 shutdown_connections 2024-04-25T16:27:37.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f81440744e0 0x7f81440769f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8164160ab0 0x7f8164163720 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d0420 0x7f8164160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 --2- 172.21.15.94:0/3227206144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81640cf900 0x7f81641503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.875+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 >> 172.21.15.94:0/3227206144 conn(0x7f81640b7b90 msgr2=0x7f816409ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.876+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 shutdown_connections 2024-04-25T16:27:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:37.876+0000 7f8169172640 1 -- 172.21.15.94:0/3227206144 wait complete. 2024-04-25T16:27:37.887 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:27:37.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:27:37.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T16:27:37.888 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T16:27:37.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T16:27:37.888 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:27:37.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:27:37.888 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:27:37.888 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T16:27:37.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:37.888 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:37.907 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::2631:683d:bbe7:78a5/64 scope link noprefixroute 2024-04-25T16:27:37.908 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:37.909 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:27:37.909 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:38.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:38.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:38.100 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:27:38.124 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T16:27:38.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:38.125 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T16:27:38.192 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T16:27:38.192 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:27:38.192 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:27:38.192 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:27:38.192 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:27:38.193 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:38.193 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:38.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:27:38.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:27:38.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:27:38.193 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:38.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:38.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:38.471 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:27:38.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:38.485 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T16:27:38.540 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:38.540 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:38.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:27:38.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T16:27:38.541 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:38.656 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T16:27:38.657 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:38.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:38.778 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:38.778 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:38.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-04-25T16:27:38.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:27:38.778 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:38.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:38.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:38.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:38.853 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T16:27:38.867 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T16:27:38.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:38.868 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T16:27:38.922 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T16:27:38.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:38.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T16:27:39.025 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T16:27:39.025 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T16:27:39.025 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-04-25T16:27:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T16:27:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T16:27:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-04-25T16:27:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-04-25T16:27:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e3c627c0-5913-4b5c-b1aa-d445103b794b" will pass to kernel 2024-04-25T16:27:39.137 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T16:27:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-04-25T16:27:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T16:27:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T16:27:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T16:27:39.139 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T16:27:39.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:39.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T16:27:39.162 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout:client4744 2024-04-25T16:27:39.257 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:27:39.258 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:39.273 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:39.368 INFO:teuthology.orchestra.run.smithi138.stdout:client4744 2024-04-25T16:27:39.369 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:39.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:39.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:39.423 INFO:teuthology.orchestra.run.smithi138.stdout:client4744 2024-04-25T16:27:39.423 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e3c627c0-5913-4b5c-b1aa-d445103b794b.client4744/status') 2024-04-25T16:27:39.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4744 (3)192.168.0.1:0/3304217427 2024-04-25T16:27:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-04-25T16:27:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-04-25T16:27:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-04-25T16:27:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:69 bytes copied, 2.3233e-05 s, 3.0 MB/s 2024-04-25T16:27:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:39.487 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T16:27:39.487 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:27:39.487 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T16:27:39.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:27:39.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.614+0000 7f7f4f827640 1 Processor -- start 2024-04-25T16:27:39.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.615+0000 7f7f4f827640 1 -- start start 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4f827640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f4809c4a0 0x7f7f480d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4f827640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7f4806e860 0x7f7f480cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4f827640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4815ea10 con 0x7f7f480d33b0 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4f827640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4806ce00 con 0x7f7f4806e860 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4f827640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4806d3c0 con 0x7f7f4809c4a0 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4ddc4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7f4806e860 0x7f7f480cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4cdc2640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7f480d33b0 0x7f7f480cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46718/0 (socket says 172.21.15.94:46718) 2024-04-25T16:27:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4cdc2640 1 -- 172.21.15.94:0/3368523559 learned_addr learned my addr 172.21.15.94:0/3368523559 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4d5c3640 1 --2- 172.21.15.94:0/3368523559 >> v2:172.21.15.138:3300/0 conn(0x7f7f4809c4a0 0x7f7f480d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4ddc4640 1 -- 172.21.15.94:0/3368523559 >> v2:172.21.15.138:3300/0 conn(0x7f7f4809c4a0 msgr2=0x7f7f480d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4ddc4640 1 --2- 172.21.15.94:0/3368523559 >> v2:172.21.15.138:3300/0 conn(0x7f7f4809c4a0 0x7f7f480d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.616+0000 7f7f4ddc4640 1 -- 172.21.15.94:0/3368523559 >> v1:172.21.15.94:6789/0 conn(0x7f7f480d33b0 legacy=0x7f7f480cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.617+0000 7f7f4ddc4640 1 -- 172.21.15.94:0/3368523559 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f480d37c0 con 0x7f7f4806e860 2024-04-25T16:27:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.617+0000 7f7f3ffff640 1 -- 172.21.15.94:0/3368523559 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861579519 0 0) 0x7f7f4815ea10 con 0x7f7f480d33b0 2024-04-25T16:27:39.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.617+0000 7f7f4ddc4640 1 --2- 172.21.15.94:0/3368523559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4806e860 0x7f7f480cfb90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f4806cf80 tx=0x7f7f4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c4ed41d858db5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:39.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.617+0000 7f7f3ffff640 1 -- 172.21.15.94:0/3368523559 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40037070 con 0x7f7f4806e860 2024-04-25T16:27:39.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.617+0000 7f7f3ffff640 1 -- 172.21.15.94:0/3368523559 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f40032440 con 0x7f7f4806e860 2024-04-25T16:27:39.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.617+0000 7f7f3ffff640 1 -- 172.21.15.94:0/3368523559 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40032810 con 0x7f7f4806e860 2024-04-25T16:27:39.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 -- 172.21.15.94:0/3368523559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4806e860 msgr2=0x7f7f480cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 --2- 172.21.15.94:0/3368523559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4806e860 0x7f7f480cfb90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f4806cf80 tx=0x7f7f4002a1a0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 -- 172.21.15.94:0/3368523559 shutdown_connections 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 --2- 172.21.15.94:0/3368523559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4806e860 0x7f7f480cfb90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 --2- 172.21.15.94:0/3368523559 >> v2:172.21.15.138:3300/0 conn(0x7f7f4809c4a0 0x7f7f480d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 -- 172.21.15.94:0/3368523559 >> 172.21.15.94:0/3368523559 conn(0x7f7f480b7bc0 msgr2=0x7f7f480b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 -- 172.21.15.94:0/3368523559 shutdown_connections 2024-04-25T16:27:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.618+0000 7f7f4f827640 1 -- 172.21.15.94:0/3368523559 wait complete. 2024-04-25T16:27:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.620+0000 7f7f4f827640 1 Processor -- start 2024-04-25T16:27:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.620+0000 7f7f4f827640 1 -- start start 2024-04-25T16:27:39.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4f827640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f4806e860 0x7f7f480c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:39.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4f827640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 0x7f7f480c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:39.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4f827640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f480d33b0 0x7f7f48160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4f827640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480b4620 con 0x7f7f4809c4a0 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4f827640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480b4340 con 0x7f7f4806e860 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4f827640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4806d050 con 0x7f7f480d33b0 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 0x7f7f480c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 0x7f7f480c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59226/0 (socket says 172.21.15.94:59226) 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4d5c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f4806e860 0x7f7f480c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 -- 172.21.15.94:0/2917497930 learned_addr learned my addr 172.21.15.94:0/2917497930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4ddc4640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f480d33b0 0x7f7f48160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 -- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f480d33b0 msgr2=0x7f7f48160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f480d33b0 0x7f7f48160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 -- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f4806e860 msgr2=0x7f7f480c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:39.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f4806e860 0x7f7f480c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 -- 172.21.15.94:0/2917497930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48160690 con 0x7f7f4809c4a0 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4d5c3640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f4806e860 0x7f7f480c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f4cdc2640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 0x7f7f480c85d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c420 tx=0x7f7f3800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.622+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380074e0 con 0x7f7f4809c4a0 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.623+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f380047e0 con 0x7f7f4809c4a0 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.623+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f480c90c0 con 0x7f7f4809c4a0 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.623+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004c10 con 0x7f7f4809c4a0 2024-04-25T16:27:39.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.623+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f480c9680 con 0x7f7f4809c4a0 2024-04-25T16:27:39.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.624+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4806a180 con 0x7f7f4809c4a0 2024-04-25T16:27:39.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.624+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f38011070 con 0x7f7f4809c4a0 2024-04-25T16:27:39.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.625+0000 7f7f3dffb640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7f2c0744b0 0x7f7f2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:39.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.625+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7f380950b0 con 0x7f7f4809c4a0 2024-04-25T16:27:39.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.625+0000 7f7f4d5c3640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7f2c0744b0 0x7f7f2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:39.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.625+0000 7f7f4d5c3640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7f2c0744b0 0x7f7f2c0769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f4809c260 tx=0x7f7f44009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:39.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.630+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f3805ee80 con 0x7f7f4809c4a0 2024-04-25T16:27:39.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.843+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7f480cebd0 con 0x7f7f4809c4a0 2024-04-25T16:27:39.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.844+0000 7f7f3dffb640 1 -- 172.21.15.94:0/2917497930 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7f38063180 con 0x7f7f4809c4a0 2024-04-25T16:27:39.845 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7f2c0744b0 msgr2=0x7f7f2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7f2c0744b0 0x7f7f2c0769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f4809c260 tx=0x7f7f44009830 comp rx=0 tx=0).stop 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 msgr2=0x7f7f480c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 0x7f7f480c85d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c420 tx=0x7f7f3800c8e0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 shutdown_connections 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7f2c0744b0 0x7f7f2c0769c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f480d33b0 0x7f7f48160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f4809c4a0 0x7f7f480c85d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 --2- 172.21.15.94:0/2917497930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f4806e860 0x7f7f480c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 >> 172.21.15.94:0/2917497930 conn(0x7f7f480b7bc0 msgr2=0x7f7f480d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 shutdown_connections 2024-04-25T16:27:39.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:39.846+0000 7f7f4f827640 1 -- 172.21.15.94:0/2917497930 wait complete. 2024-04-25T16:27:39.857 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:27:39.857 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:27:39.857 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T16:27:39.858 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T16:27:39.858 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T16:27:39.858 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:27:39.858 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:27:39.858 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:27:39.858 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T16:27:39.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:39.858 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T16:27:39.874 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:39.874 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:39.874 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:39.874 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:39.874 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-04-25T16:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::2631:683d:bbe7:78a5/64 scope link noprefixroute 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 26:1e:6b:68:87:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::a4e7:20ff:fe45:ef13/64 scope link 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 26:1e:6b:68:87:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::241e:6bff:fe68:8713/64 scope link tentative 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:39.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:39.877 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:27:39.937 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T16:27:39.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:39.939 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T16:27:39.995 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T16:27:39.995 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:27:39.995 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:27:39.996 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:27:39.996 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:27:39.996 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:39.996 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:39.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:27:39.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:27:39.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:27:39.996 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:40.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:40.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:40.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:40.119 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:27:40.134 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T16:27:40.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:40.134 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T16:27:40.188 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T16:27:40.189 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:40.189 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:40.189 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:27:40.189 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T16:27:40.189 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:40.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:40.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:40.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:40.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 92:e4:3d:6a:18:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:27:40.326 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.327 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::90e4:3dff:fe6a:18b1/64 scope link tentative 2024-04-25T16:27:40.327 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:40.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.391 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:27:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 92:e4:3d:6a:18:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:27:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:27:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::90e4:3dff:fe6a:18b1/64 scope link tentative 2024-04-25T16:27:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:27:40.393 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T16:27:40.393 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:40.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:40.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:40.540 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:27:40.540 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:27:40.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2024-04-25T16:27:40.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T16:27:40.540 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:27:40.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:40.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:40.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:40.616 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T16:27:40.630 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T16:27:40.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:27:40.630 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T16:27:40.685 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T16:27:40.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:40.685 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T16:27:40.779 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T16:27:40.779 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T16:27:40.779 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-04-25T16:27:40.789 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T16:27:40.790 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T16:27:40.790 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-04-25T16:27:40.790 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:40.790 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:40.800 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T16:27:40.800 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T16:27:40.800 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-04-25T16:27:40.800 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-04-25T16:27:40.800 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=e3c627c0-5913-4b5c-b1aa-d445103b794b" will pass to kernel 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T16:27:40.801 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T16:27:40.802 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T16:27:40.803 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T16:27:40.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:40.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T16:27:40.826 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:40.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:40.921 INFO:teuthology.orchestra.run.smithi138.stdout:client4848 2024-04-25T16:27:40.921 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:27:40.921 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:40.937 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:41.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:41.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:41.032 INFO:teuthology.orchestra.run.smithi138.stdout:client4848 2024-04-25T16:27:41.032 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:27:41.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:27:41.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:41.087 INFO:teuthology.orchestra.run.smithi138.stdout:client4848 2024-04-25T16:27:41.087 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e3c627c0-5913-4b5c-b1aa-d445103b794b.client4848/status') 2024-04-25T16:27:41.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:27:41.143 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4848 (3)172.21.15.138:0/299590455 2024-04-25T16:27:41.143 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-04-25T16:27:41.143 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-04-25T16:27:41.144 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-04-25T16:27:41.144 INFO:teuthology.orchestra.run.smithi138.stderr:70 bytes copied, 2.1982e-05 s, 3.2 MB/s 2024-04-25T16:27:41.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:27:41.150 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T16:27:41.160 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T16:27:41.160 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:27:41.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'cd $TESTDIR/mnt.0 && dd if=/dev/zero of=./foo count=100' 2024-04-25T16:27:41.195 INFO:teuthology.orchestra.run.smithi094.stderr:100+0 records in 2024-04-25T16:27:41.195 INFO:teuthology.orchestra.run.smithi094.stderr:100+0 records out 2024-04-25T16:27:41.195 INFO:teuthology.orchestra.run.smithi094.stderr:51200 bytes (51 kB, 50 KiB) copied, 0.00259866 s, 19.7 MB/s 2024-04-25T16:27:41.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sleep 2' 2024-04-25T16:27:43.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'cd $TESTDIR/mnt.0 && truncate --size 0 ./foo' 2024-04-25T16:27:43.309 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T16:27:43.321 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T16:27:43.321 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:43.322 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:43.367 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:43.367 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:43.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:43.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.580+0000 7f990ceaf640 1 Processor -- start 2024-04-25T16:27:43.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.580+0000 7f990ceaf640 1 -- start start 2024-04-25T16:27:43.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f990ceaf640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f99080b37c0 0x7f99080cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f990ceaf640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f99080cda90 0x7f99080d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f990ceaf640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9908160db0 con 0x7f99080d2bc0 2024-04-25T16:27:43.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f990ceaf640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f99080ce1b0 con 0x7f99080b37c0 2024-04-25T16:27:43.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f990ceaf640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f99080ce490 con 0x7f99080cda90 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f990659c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f99080b37c0 0x7f99080cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f9906d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f99080d2bc0 0x7f99080d2fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44348/0 (socket says 172.21.15.94:44348) 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.581+0000 7f9906d9d640 1 -- 172.21.15.94:0/2343975518 learned_addr learned my addr 172.21.15.94:0/2343975518 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f98fffff640 1 --2- 172.21.15.94:0/2343975518 >> v2:172.21.15.138:3300/0 conn(0x7f99080cda90 0x7f99080d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f990659c640 1 -- 172.21.15.94:0/2343975518 >> v2:172.21.15.138:3300/0 conn(0x7f99080cda90 msgr2=0x7f99080d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f990659c640 1 --2- 172.21.15.94:0/2343975518 >> v2:172.21.15.138:3300/0 conn(0x7f99080cda90 0x7f99080d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f990659c640 1 -- 172.21.15.94:0/2343975518 >> v1:172.21.15.94:6789/0 conn(0x7f99080d2bc0 legacy=0x7f99080d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f990659c640 1 -- 172.21.15.94:0/2343975518 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99080ce770 con 0x7f99080b37c0 2024-04-25T16:27:43.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f990659c640 1 --2- 172.21.15.94:0/2343975518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f99080cd550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f99080a0fc0 tx=0x7f98f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad397c3a7d3983d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.582+0000 7f9905d9b640 1 -- 172.21.15.94:0/2343975518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8037070 con 0x7f99080b37c0 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f9905d9b640 1 -- 172.21.15.94:0/2343975518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f8032440 con 0x7f99080b37c0 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f9905d9b640 1 -- 172.21.15.94:0/2343975518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8032810 con 0x7f99080b37c0 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 -- 172.21.15.94:0/2343975518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 msgr2=0x7f99080cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 --2- 172.21.15.94:0/2343975518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f99080cd550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f99080a0fc0 tx=0x7f98f802a1a0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 -- 172.21.15.94:0/2343975518 shutdown_connections 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 --2- 172.21.15.94:0/2343975518 >> v2:172.21.15.138:3300/0 conn(0x7f99080cda90 0x7f99080d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 --2- 172.21.15.94:0/2343975518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f99080cd550 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 -- 172.21.15.94:0/2343975518 >> 172.21.15.94:0/2343975518 conn(0x7f99080b7bc0 msgr2=0x7f99080b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:43.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 -- 172.21.15.94:0/2343975518 shutdown_connections 2024-04-25T16:27:43.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.583+0000 7f990ceaf640 1 -- 172.21.15.94:0/2343975518 wait complete. 2024-04-25T16:27:43.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.585+0000 7f990ceaf640 1 Processor -- start 2024-04-25T16:27:43.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.585+0000 7f990ceaf640 1 -- start start 2024-04-25T16:27:43.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.586+0000 7f990ceaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f990809f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.586+0000 7f990ceaf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99080cda90 0x7f9908159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.586+0000 7f990ceaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99080d2bc0 0x7f99080b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.586+0000 7f990ceaf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99080d20f0 con 0x7f99080b37c0 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.586+0000 7f990ceaf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99080cdea0 con 0x7f99080d2bc0 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.586+0000 7f990ceaf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f990806f7f0 con 0x7f99080cda90 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f990809f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f990809f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52468/0 (socket says 172.21.15.94:52468) 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 -- 172.21.15.94:0/443289267 learned_addr learned my addr 172.21.15.94:0/443289267 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 -- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99080cda90 msgr2=0x7f9908159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99080cda90 0x7f9908159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 -- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99080d2bc0 msgr2=0x7f99080b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f9906d9d640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99080d2bc0 0x7f99080b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99080d2bc0 0x7f99080b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 -- 172.21.15.94:0/443289267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990809fdb0 con 0x7f99080b37c0 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990659c640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f990809f870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f98f8000c00 tx=0x7f98f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:43.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8037040 con 0x7f99080b37c0 2024-04-25T16:27:43.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f8042070 con 0x7f99080b37c0 2024-04-25T16:27:43.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99080a02c0 con 0x7f99080b37c0 2024-04-25T16:27:43.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8030bd0 con 0x7f99080b37c0 2024-04-25T16:27:43.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.587+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99080a0880 con 0x7f99080b37c0 2024-04-25T16:27:43.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.588+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990806a180 con 0x7f99080b37c0 2024-04-25T16:27:43.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.589+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98f8047460 con 0x7f99080b37c0 2024-04-25T16:27:43.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.589+0000 7f98feffd640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f98e80744b0 0x7f98e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.590+0000 7f98fffff640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f98e80744b0 0x7f98e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.590+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f98f80b6280 con 0x7f99080b37c0 2024-04-25T16:27:43.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.590+0000 7f98fffff640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f98e80744b0 0x7f98e80769c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fd0 tx=0x7f98f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.594+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98f8032e20 con 0x7f99080b37c0 2024-04-25T16:27:43.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.772+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99080b31e0 con 0x7f99080b37c0 2024-04-25T16:27:43.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.774+0000 7f98feffd640 1 -- 172.21.15.94:0/443289267 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18868 (secure 0 0 0) 0x7f98f8080610 con 0x7f99080b37c0 2024-04-25T16:27:43.774 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:43.774 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:36.681885+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f98e80744b0 msgr2=0x7f98e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f98e80744b0 0x7f98e80769c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fd0 tx=0x7f98f0008040 comp rx=0 tx=0).stop 2024-04-25T16:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 msgr2=0x7f990809f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f990809f870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f98f8000c00 tx=0x7f98f80046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 shutdown_connections 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f98e80744b0 0x7f98e80769c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99080d2bc0 0x7f99080b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99080cda90 0x7f9908159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 --2- 172.21.15.94:0/443289267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99080b37c0 0x7f990809f870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 >> 172.21.15.94:0/443289267 conn(0x7f99080b7bc0 msgr2=0x7f99081602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 shutdown_connections 2024-04-25T16:27:43.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.776+0000 7f990ceaf640 1 -- 172.21.15.94:0/443289267 wait complete. 2024-04-25T16:27:43.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:43.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.897+0000 7f410c38f640 1 Processor -- start 2024-04-25T16:27:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.897+0000 7f410c38f640 1 -- start start 2024-04-25T16:27:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.898+0000 7f410c38f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41040b9780 0x7f410409c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.898+0000 7f410c38f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f410409c7f0 0x7f410406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410c38f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040d39e0 con 0x7f410406e880 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410c38f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f41040d2dd0 con 0x7f410409c7f0 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410c38f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41040d30b0 con 0x7f41040b9780 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f410409c7f0 0x7f410406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f410409c7f0 0x7f410406e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52476/0 (socket says 172.21.15.94:52476) 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 -- 172.21.15.94:0/1164947001 learned_addr learned my addr 172.21.15.94:0/1164947001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 -- 172.21.15.94:0/1164947001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41040b9780 msgr2=0x7f410409c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410a12b640 1 --2- 172.21.15.94:0/1164947001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41040b9780 0x7f410409c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 --2- 172.21.15.94:0/1164947001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41040b9780 0x7f410409c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 -- 172.21.15.94:0/1164947001 >> v1:172.21.15.138:6789/0 conn(0x7f410406e880 legacy=0x7f4104164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.899+0000 7f410992a640 1 -- 172.21.15.94:0/1164947001 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040d3390 con 0x7f410409c7f0 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.900+0000 7f410992a640 1 --2- 172.21.15.94:0/1164947001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f410409c7f0 0x7f410406e340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009870 tx=0x7f40f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420dd5bb2c5fc368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.900+0000 7f4109129640 1 -- 172.21.15.94:0/1164947001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4037070 con 0x7f410409c7f0 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.900+0000 7f4109129640 1 -- 172.21.15.94:0/1164947001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f4032440 con 0x7f410409c7f0 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.900+0000 7f4109129640 1 -- 172.21.15.94:0/1164947001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4032810 con 0x7f410409c7f0 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.900+0000 7f410c38f640 1 -- 172.21.15.94:0/1164947001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f410409c7f0 msgr2=0x7f410406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.900+0000 7f410c38f640 1 --2- 172.21.15.94:0/1164947001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f410409c7f0 0x7f410406e340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009870 tx=0x7f40f402a210 comp rx=0 tx=0).stop 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.901+0000 7f410c38f640 1 -- 172.21.15.94:0/1164947001 shutdown_connections 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.901+0000 7f410c38f640 1 --2- 172.21.15.94:0/1164947001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f410409c7f0 0x7f410406e340 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.901+0000 7f410c38f640 1 --2- 172.21.15.94:0/1164947001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41040b9780 0x7f410409c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.901+0000 7f410c38f640 1 -- 172.21.15.94:0/1164947001 >> 172.21.15.94:0/1164947001 conn(0x7f41040b5b20 msgr2=0x7f41040b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.901+0000 7f410c38f640 1 -- 172.21.15.94:0/1164947001 shutdown_connections 2024-04-25T16:27:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.901+0000 7f410c38f640 1 -- 172.21.15.94:0/1164947001 wait complete. 2024-04-25T16:27:43.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.902+0000 7f410c38f640 1 Processor -- start 2024-04-25T16:27:43.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.902+0000 7f410c38f640 1 -- start start 2024-04-25T16:27:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410406e880 0x7f41040a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f410409c7f0 0x7f41040b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 0x7f4104154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41040b50e0 con 0x7f41040b9780 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41040b4f60 con 0x7f410409c7f0 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f410409d040 con 0x7f410406e880 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 0x7f4104154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 0x7f4104154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52488/0 (socket says 172.21.15.94:52488) 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 -- 172.21.15.94:0/1544221457 learned_addr learned my addr 172.21.15.94:0/1544221457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a12b640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410406e880 0x7f41040a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410992a640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f410409c7f0 0x7f41040b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 -- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410406e880 msgr2=0x7f41040a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410406e880 0x7f41040a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 -- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f410409c7f0 msgr2=0x7f41040b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f410409c7f0 0x7f41040b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 -- 172.21.15.94:0/1544221457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4104154e00 con 0x7f41040b9780 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410992a640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f410409c7f0 0x7f41040b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410a92c640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 0x7f4104154310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00be60 tx=0x7f40fc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc019070 con 0x7f41040b9780 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc014440 con 0x7f41040b9780 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410409ddd0 con 0x7f41040b9780 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.904+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc014870 con 0x7f41040b9780 2024-04-25T16:27:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.905+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41040b4870 con 0x7f41040b9780 2024-04-25T16:27:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.906+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41040b4300 con 0x7f41040b9780 2024-04-25T16:27:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.906+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f40fc01c070 con 0x7f41040b9780 2024-04-25T16:27:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.906+0000 7f40faffd640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f40e80744b0 0x7f40e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.906+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f40fc0955f0 con 0x7f41040b9780 2024-04-25T16:27:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.906+0000 7f410a12b640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f40e80744b0 0x7f40e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.907+0000 7f410a12b640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f40e80744b0 0x7f40e80769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4100006fd0 tx=0x7f4100008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:43.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:43.911+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40fc05f980 con 0x7f41040b9780 2024-04-25T16:27:44.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.085+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f410406a180 con 0x7f41040b9780 2024-04-25T16:27:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.085+0000 7f40faffd640 1 -- 172.21.15.94:0/1544221457 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f40fc05f280 con 0x7f41040b9780 2024-04-25T16:27:44.086 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f40e80744b0 msgr2=0x7f40e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f40e80744b0 0x7f40e80769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4100006fd0 tx=0x7f4100008040 comp rx=0 tx=0).stop 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 msgr2=0x7f4104154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 0x7f4104154310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00be60 tx=0x7f40fc00bf50 comp rx=0 tx=0).stop 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 shutdown_connections 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f40e80744b0 0x7f40e80769c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41040b9780 0x7f4104154310 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f410409c7f0 0x7f41040b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 --2- 172.21.15.94:0/1544221457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410406e880 0x7f41040a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 >> 172.21.15.94:0/1544221457 conn(0x7f41040b5b20 msgr2=0x7f4104164680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:44.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 shutdown_connections 2024-04-25T16:27:44.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.087+0000 7f410c38f640 1 -- 172.21.15.94:0/1544221457 wait complete. 2024-04-25T16:27:44.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:44.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.211+0000 7f0aafff3640 1 Processor -- start 2024-04-25T16:27:44.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.211+0000 7f0aafff3640 1 -- start start 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aafff3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa806d450 0x7f0aa815f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aafff3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0aa815f960 0x7f0aa80cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aafff3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa806df40 con 0x7f0aa806f7e0 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aafff3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa815ef90 con 0x7f0aa815f960 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aafff3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa815ec40 con 0x7f0aa806d450 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aae590640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0aa815f960 0x7f0aa80cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aadd8f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0aa806f7e0 0x7f0aa80d16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44362/0 (socket says 172.21.15.94:44362) 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.212+0000 7f0aadd8f640 1 -- 172.21.15.94:0/3244619194 learned_addr learned my addr 172.21.15.94:0/3244619194 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aad58e640 1 --2- 172.21.15.94:0/3244619194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa806d450 0x7f0aa815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aae590640 1 -- 172.21.15.94:0/3244619194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa806d450 msgr2=0x7f0aa815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aae590640 1 --2- 172.21.15.94:0/3244619194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa806d450 0x7f0aa815f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aae590640 1 -- 172.21.15.94:0/3244619194 >> v1:172.21.15.94:6789/0 conn(0x7f0aa806f7e0 legacy=0x7f0aa80d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aae590640 1 -- 172.21.15.94:0/3244619194 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa806d860 con 0x7f0aa815f960 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aacd8d640 1 -- 172.21.15.94:0/3244619194 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677353206 0 0) 0x7f0aa806df40 con 0x7f0aa806f7e0 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aae590640 1 --2- 172.21.15.94:0/3244619194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa815f960 0x7f0aa80cf0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0aa815f110 tx=0x7f0aa002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53ae24c64d60b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.213+0000 7f0aacd8d640 1 -- 172.21.15.94:0/3244619194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa002ae60 con 0x7f0aa815f960 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aacd8d640 1 -- 172.21.15.94:0/3244619194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa0007740 con 0x7f0aa815f960 2024-04-25T16:27:44.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aacd8d640 1 -- 172.21.15.94:0/3244619194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0007b10 con 0x7f0aa815f960 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 -- 172.21.15.94:0/3244619194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa815f960 msgr2=0x7f0aa80cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3244619194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa815f960 0x7f0aa80cf0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0aa815f110 tx=0x7f0aa002a640 comp rx=0 tx=0).stop 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 -- 172.21.15.94:0/3244619194 shutdown_connections 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3244619194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa815f960 0x7f0aa80cf0a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3244619194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa806d450 0x7f0aa815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 -- 172.21.15.94:0/3244619194 >> 172.21.15.94:0/3244619194 conn(0x7f0aa80b4270 msgr2=0x7f0aa80ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 -- 172.21.15.94:0/3244619194 shutdown_connections 2024-04-25T16:27:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.214+0000 7f0aafff3640 1 -- 172.21.15.94:0/3244619194 wait complete. 2024-04-25T16:27:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.216+0000 7f0aafff3640 1 Processor -- start 2024-04-25T16:27:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.216+0000 7f0aafff3640 1 -- start start 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aafff3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa806d450 0x7f0aa814bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aafff3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 0x7f0aa80cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aafff3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa815f960 0x7f0aa80ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aafff3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8070320 con 0x7f0aa806f7e0 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aafff3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8070040 con 0x7f0aa806d450 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aafff3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa80cf5e0 con 0x7f0aa815f960 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aad58e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 0x7f0aa80cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aad58e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 0x7f0aa80cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52504/0 (socket says 172.21.15.94:52504) 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.217+0000 7f0aad58e640 1 -- 172.21.15.94:0/3039323210 learned_addr learned my addr 172.21.15.94:0/3039323210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:44.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aad58e640 1 -- 172.21.15.94:0/3039323210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa815f960 msgr2=0x7f0aa80ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aad58e640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa815f960 0x7f0aa80ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aad58e640 1 -- 172.21.15.94:0/3039323210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa806d450 msgr2=0x7f0aa814bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aad58e640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa806d450 0x7f0aa814bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aad58e640 1 -- 172.21.15.94:0/3039323210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa814c920 con 0x7f0aa806f7e0 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aad58e640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 0x7f0aa80cddc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c820 tx=0x7f0a9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98015070 con 0x7f0aa806f7e0 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a98004020 con 0x7f0aa806f7e0 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa8149d00 con 0x7f0aa806f7e0 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980043f0 con 0x7f0aa806f7e0 2024-04-25T16:27:44.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.218+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa814a2c0 con 0x7f0aa806f7e0 2024-04-25T16:27:44.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.219+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa806a180 con 0x7f0aa806f7e0 2024-04-25T16:27:44.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.220+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0a98007830 con 0x7f0aa806f7e0 2024-04-25T16:27:44.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.220+0000 7f0a9e7fc640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0a8c0744b0 0x7f0a8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.220+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0a980952f0 con 0x7f0aa806f7e0 2024-04-25T16:27:44.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.220+0000 7f0aadd8f640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0a8c0744b0 0x7f0a8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.220+0000 7f0aadd8f640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0a8c0744b0 0x7f0a8c0769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007fd0 tx=0x7f0aa4007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.225+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a9805f7a0 con 0x7f0aa806f7e0 2024-04-25T16:27:44.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.397+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0aa80afcf0 con 0x7f0aa806f7e0 2024-04-25T16:27:44.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.397+0000 7f0a9e7fc640 1 -- 172.21.15.94:0/3039323210 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0a9805f0a0 con 0x7f0aa806f7e0 2024-04-25T16:27:44.398 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:44.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0a8c0744b0 msgr2=0x7f0a8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0a8c0744b0 0x7f0a8c0769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007fd0 tx=0x7f0aa4007960 comp rx=0 tx=0).stop 2024-04-25T16:27:44.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 msgr2=0x7f0aa80cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 0x7f0aa80cddc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c820 tx=0x7f0a9800cce0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 shutdown_connections 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0a8c0744b0 0x7f0a8c0769c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0aa815f960 0x7f0aa80ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa806f7e0 0x7f0aa80cddc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 --2- 172.21.15.94:0/3039323210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa806d450 0x7f0aa814bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 >> 172.21.15.94:0/3039323210 conn(0x7f0aa80b4270 msgr2=0x7f0aa806ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.399+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 shutdown_connections 2024-04-25T16:27:44.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.400+0000 7f0aafff3640 1 -- 172.21.15.94:0/3039323210 wait complete. 2024-04-25T16:27:44.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.520+0000 7fc48ebee640 1 Processor -- start 2024-04-25T16:27:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.520+0000 7fc48ebee640 1 -- start start 2024-04-25T16:27:44.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc48ebee640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc48815d740 0x7fc48809fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc48ebee640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4880707b0 con 0x7fc4880703a0 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc48ebee640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48806d930 con 0x7fc4880cf7b0 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc48ebee640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4880d17e0 con 0x7fc48815d740 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc487fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc4880703a0 0x7fc4880cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44378/0 (socket says 172.21.15.94:44378) 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc487fff640 1 -- 172.21.15.94:0/1855465096 learned_addr learned my addr 172.21.15.94:0/1855465096 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc48c98a640 1 --2- 172.21.15.94:0/1855465096 >> v2:172.21.15.138:3300/0 conn(0x7fc48815d740 0x7fc48809fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673118891 0 0) 0x7fc4880707b0 con 0x7fc4880703a0 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc470003520 con 0x7fc4880703a0 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1524039081 0 0) 0x7fc470003520 con 0x7fc4880703a0 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 >> v2:172.21.15.138:3300/0 conn(0x7fc48815d740 msgr2=0x7fc48809fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 --2- 172.21.15.94:0/1855465096 >> v2:172.21.15.138:3300/0 conn(0x7fc48815d740 0x7fc48809fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 >> v1:172.21.15.138:6789/0 conn(0x7fc4880cf7b0 legacy=0x7fc4880b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880d1ac0 con 0x7fc4880703a0 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fc47c001670 con 0x7fc4880703a0 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 >> v1:172.21.15.94:6789/0 conn(0x7fc4880703a0 legacy=0x7fc4880cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc470003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.522+0000 7fc4877fe640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc470004800 0x7fc470004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc4877fe640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4700051c0 0x7fc47000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880d17e0 con 0x7fc4700039b0 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc47000c160 con 0x7fc4700051c0 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc47000c360 con 0x7fc470004800 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48c98a640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc470003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc487fff640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc470004800 0x7fc470004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48d18b640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4700051c0 0x7fc47000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48c98a640 1 -- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc470004800 msgr2=0x7fc470004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48c98a640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc470004800 0x7fc470004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48c98a640 1 -- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4700051c0 msgr2=0x7fc47000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48c98a640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4700051c0 0x7fc47000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48d18b640 1 -- 172.21.15.94:0/1855465096 reap_dead start 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.523+0000 7fc48c98a640 1 -- 172.21.15.94:0/1855465096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc470003520 con 0x7fc4700039b0 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48c98a640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc470003e20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc478009a90 tx=0x7fc470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca3028d516dd67d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478036070 con 0x7fc4700039b0 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc478031400 con 0x7fc4700039b0 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc4877fe640 1 -- 172.21.15.94:0/1855465096 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478009690 con 0x7fc4700039b0 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 -- 172.21.15.94:0/1855465096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 msgr2=0x7fc470003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc470003e20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc478009a90 tx=0x7fc470005830 comp rx=0 tx=0).stop 2024-04-25T16:27:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 -- 172.21.15.94:0/1855465096 shutdown_connections 2024-04-25T16:27:44.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 --2- 172.21.15.94:0/1855465096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc470003e20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 -- 172.21.15.94:0/1855465096 >> 172.21.15.94:0/1855465096 conn(0x7fc4880b4270 msgr2=0x7fc4880ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:44.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 -- 172.21.15.94:0/1855465096 shutdown_connections 2024-04-25T16:27:44.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.524+0000 7fc48ebee640 1 -- 172.21.15.94:0/1855465096 wait complete. 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.526+0000 7fc48ebee640 1 Processor -- start 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.526+0000 7fc48ebee640 1 -- start start 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc48814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc48814dbf0 0x7fc4880aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4880af460 0x7fc488150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc48809fdf0 con 0x7fc4700039b0 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc48809fc70 con 0x7fc4880af460 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc48809ff70 con 0x7fc48814dbf0 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc48814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc48814d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52508/0 (socket says 172.21.15.94:52508) 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 -- 172.21.15.94:0/849288986 learned_addr learned my addr 172.21.15.94:0/849288986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc487fff640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc48814dbf0 0x7fc4880aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48d18b640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4880af460 0x7fc488150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 -- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc48814dbf0 msgr2=0x7fc4880aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc48814dbf0 0x7fc4880aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 -- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4880af460 msgr2=0x7fc488150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4880af460 0x7fc488150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 -- 172.21.15.94:0/849288986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880af870 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48c98a640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc48814d340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc478009a90 tx=0x7fc478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478040020 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47803b680 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48809f720 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478036040 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.528+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48815ded0 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.529+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc488071e20 con 0x7fc4700039b0 2024-04-25T16:27:44.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.530+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc478043020 con 0x7fc4700039b0 2024-04-25T16:27:44.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.530+0000 7fc4857fa640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc4740744e0 0x7fc4740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.531+0000 7fc487fff640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc4740744e0 0x7fc4740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.531+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc4780b5e10 con 0x7fc4700039b0 2024-04-25T16:27:44.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.531+0000 7fc487fff640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc4740744e0 0x7fc4740769f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc488063980 tx=0x7fc47c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.535+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc488071e20 con 0x7fc4700039b0 2024-04-25T16:27:44.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.714+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc48806a180 con 0x7fc4700039b0 2024-04-25T16:27:44.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.715+0000 7fc4857fa640 1 -- 172.21.15.94:0/849288986 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc47804a030 con 0x7fc4700039b0 2024-04-25T16:27:44.715 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc4740744e0 msgr2=0x7fc4740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc4740744e0 0x7fc4740769f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc488063980 tx=0x7fc47c009040 comp rx=0 tx=0).stop 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 msgr2=0x7fc48814d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc48814d340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc478009a90 tx=0x7fc478004730 comp rx=0 tx=0).stop 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 shutdown_connections 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc4740744e0 0x7fc4740769f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4880af460 0x7fc488150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc48814dbf0 0x7fc4880aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 --2- 172.21.15.94:0/849288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4700039b0 0x7fc48814d340 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 >> 172.21.15.94:0/849288986 conn(0x7fc4880b4270 msgr2=0x7fc48806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:44.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 shutdown_connections 2024-04-25T16:27:44.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.717+0000 7fc48ebee640 1 -- 172.21.15.94:0/849288986 wait complete. 2024-04-25T16:27:44.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:44.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.838+0000 7fc053e9f640 1 Processor -- start 2024-04-25T16:27:44.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.838+0000 7fc053e9f640 1 -- start start 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.839+0000 7fc053e9f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 0x7fc04c0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.839+0000 7fc053e9f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c0cf6d0 con 0x7fc04c0d3640 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.839+0000 7fc053e9f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c0cf9b0 con 0x7fc04c0b7440 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.839+0000 7fc053e9f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c0cfc90 con 0x7fc04c0b97b0 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.839+0000 7fc05143a640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc04c0d3640 0x7fc04c0cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44388/0 (socket says 172.21.15.94:44388) 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.839+0000 7fc05143a640 1 -- 172.21.15.94:0/666806431 learned_addr learned my addr 172.21.15.94:0/666806431 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349230233 0 0) 0x7fc04c0cf6d0 con 0x7fc04c0d3640 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc038003520 con 0x7fc04c0d3640 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc051c3b640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 0x7fc04c0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 446767829 0 0) 0x7fc038003520 con 0x7fc04c0d3640 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 msgr2=0x7fc04c0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 0x7fc04c0d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 >> v1:172.21.15.138:6789/0 conn(0x7fc04c0b7440 legacy=0x7fc04c0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c0d2900 con 0x7fc04c0d3640 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fc03c001670 con 0x7fc04c0d3640 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 >> v1:172.21.15.94:6789/0 conn(0x7fc04c0d3640 legacy=0x7fc04c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380039e0 0x7fc038003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc038004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 0x7fc038005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c0cfc90 con 0x7fc038004800 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc03800baf0 con 0x7fc0380039e0 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc03800bcf0 con 0x7fc04c0b97b0 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05143a640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc038004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc051c3b640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380039e0 0x7fc038003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05243c640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 0x7fc038005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05143a640 1 -- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 msgr2=0x7fc038005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05143a640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c0b97b0 0x7fc038005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05143a640 1 -- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380039e0 msgr2=0x7fc038003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05143a640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380039e0 0x7fc038003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc05143a640 1 -- 172.21.15.94:0/666806431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc038003520 con 0x7fc038004800 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.840+0000 7fc051c3b640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380039e0 0x7fc038003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.841+0000 7fc05143a640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc038004c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc04c074a20 tx=0x7fc038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b053e5a9ea0d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.841+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c007d30 con 0x7fc038004800 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.841+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c005ad0 con 0x7fc038004800 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.841+0000 7fc050c39640 1 -- 172.21.15.94:0/666806431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c007070 con 0x7fc038004800 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc053e9f640 1 -- 172.21.15.94:0/666806431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 msgr2=0x7fc038004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc053e9f640 1 --2- 172.21.15.94:0/666806431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc038004c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc04c074a20 tx=0x7fc038005830 comp rx=0 tx=0).stop 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc05243c640 1 -- 172.21.15.94:0/666806431 reap_dead start 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc053e9f640 1 -- 172.21.15.94:0/666806431 shutdown_connections 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc053e9f640 1 -- 172.21.15.94:0/666806431 >> 172.21.15.94:0/666806431 conn(0x7fc04c0d3fa0 msgr2=0x7fc04c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc053e9f640 1 -- 172.21.15.94:0/666806431 shutdown_connections 2024-04-25T16:27:44.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.842+0000 7fc053e9f640 1 -- 172.21.15.94:0/666806431 wait complete. 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.844+0000 7fc053e9f640 1 Processor -- start 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.844+0000 7fc053e9f640 1 -- start start 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc053e9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc04c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc053e9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1e9aa0 0x7fc04c1e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc053e9f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c15c690 0x7fc04c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc053e9f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c162360 con 0x7fc038004800 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc053e9f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c0b9bc0 con 0x7fc04c1e9aa0 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc053e9f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c0cf340 con 0x7fc04c15c690 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc04c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc04c0ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52528/0 (socket says 172.21.15.94:52528) 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 -- 172.21.15.94:0/1296743538 learned_addr learned my addr 172.21.15.94:0/1296743538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:44.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc05143a640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1e9aa0 0x7fc04c1e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc05243c640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c15c690 0x7fc04c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 -- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c15c690 msgr2=0x7fc04c15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c15c690 0x7fc04c15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 -- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1e9aa0 msgr2=0x7fc04c1e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1e9aa0 0x7fc04c1e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.845+0000 7fc051c3b640 1 -- 172.21.15.94:0/1296743538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c15cfe0 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.846+0000 7fc051c3b640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc04c0ca9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc048011700 tx=0x7fc048011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.846+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04800f9c0 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.846+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04800fb60 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.846+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c1e8a00 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.846+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048015460 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.846+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1e8fc0 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.847+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c06a180 con 0x7fc038004800 2024-04-25T16:27:44.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.847+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc048015600 con 0x7fc038004800 2024-04-25T16:27:44.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.848+0000 7fc0427fc640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0300744b0 0x7fc0300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:44.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.848+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc048096070 con 0x7fc038004800 2024-04-25T16:27:44.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.848+0000 7fc05143a640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0300744b0 0x7fc0300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:44.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.848+0000 7fc05143a640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0300744b0 0x7fc0300769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc03c032000 tx=0x7fc03c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:44.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:44.853+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc04c15cfe0 con 0x7fc038004800 2024-04-25T16:27:45.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.028+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc04c0d2380 con 0x7fc038004800 2024-04-25T16:27:45.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.029+0000 7fc0427fc640 1 -- 172.21.15.94:0/1296743538 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc048019030 con 0x7fc038004800 2024-04-25T16:27:45.029 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0300744b0 msgr2=0x7fc0300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0300744b0 0x7fc0300769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc03c032000 tx=0x7fc03c009030 comp rx=0 tx=0).stop 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 msgr2=0x7fc04c0ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc04c0ca9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc048011700 tx=0x7fc048011bc0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 shutdown_connections 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fc0300744b0 0x7fc0300769c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc04c15c690 0x7fc04c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1e9aa0 0x7fc04c1e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 --2- 172.21.15.94:0/1296743538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc038004800 0x7fc04c0ca9d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 >> 172.21.15.94:0/1296743538 conn(0x7fc04c0d3fa0 msgr2=0x7fc04c0b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 shutdown_connections 2024-04-25T16:27:45.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.031+0000 7fc053e9f640 1 -- 172.21.15.94:0/1296743538 wait complete. 2024-04-25T16:27:45.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:27:45.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.150+0000 7f7664f6c640 1 Processor -- start 2024-04-25T16:27:45.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.150+0000 7f7664f6c640 1 -- start start 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f7664f6c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008da90 0x7f76600cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f7664f6c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f76600cf460 0x7f76600cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f7664f6c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76600b4a40 con 0x7f766008e3a0 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f7664f6c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f76600b9710 con 0x7f76600cf460 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f7664f6c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76600b4480 con 0x7f766008da90 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f76600cf460 0x7f76600cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f76600cf460 0x7f76600cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52542/0 (socket says 172.21.15.94:52542) 2024-04-25T16:27:45.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 -- 172.21.15.94:0/2878314474 learned_addr learned my addr 172.21.15.94:0/2878314474 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:45.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765dd9b640 1 --2- 172.21.15.94:0/2878314474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008da90 0x7f76600cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 -- 172.21.15.94:0/2878314474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008da90 msgr2=0x7f76600cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:45.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 --2- 172.21.15.94:0/2878314474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008da90 0x7f76600cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 -- 172.21.15.94:0/2878314474 >> v1:172.21.15.138:6789/0 conn(0x7f766008e3a0 legacy=0x7f766008d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:45.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.152+0000 7f765ed9d640 1 -- 172.21.15.94:0/2878314474 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76600b4760 con 0x7f76600cf460 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.153+0000 7f765ed9d640 1 --2- 172.21.15.94:0/2878314474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cf870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7658009870 tx=0x7f765802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10390997e091ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.153+0000 7f765d59a640 1 -- 172.21.15.94:0/2878314474 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658037070 con 0x7f76600cf460 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.153+0000 7f765d59a640 1 -- 172.21.15.94:0/2878314474 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658032440 con 0x7f76600cf460 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.153+0000 7f765d59a640 1 -- 172.21.15.94:0/2878314474 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658032810 con 0x7f76600cf460 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 -- 172.21.15.94:0/2878314474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 msgr2=0x7f76600cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 --2- 172.21.15.94:0/2878314474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cf870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7658009870 tx=0x7f765802a210 comp rx=0 tx=0).stop 2024-04-25T16:27:45.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 -- 172.21.15.94:0/2878314474 shutdown_connections 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 --2- 172.21.15.94:0/2878314474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cf870 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 --2- 172.21.15.94:0/2878314474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008da90 0x7f76600cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 -- 172.21.15.94:0/2878314474 >> 172.21.15.94:0/2878314474 conn(0x7f76600b5a60 msgr2=0x7f76600b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 -- 172.21.15.94:0/2878314474 shutdown_connections 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.154+0000 7f7664f6c640 1 -- 172.21.15.94:0/2878314474 wait complete. 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.156+0000 7f7664f6c640 1 Processor -- start 2024-04-25T16:27:45.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.156+0000 7f7664f6c640 1 -- start start 2024-04-25T16:27:45.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f7664f6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f766008da90 0x7f76600a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f7664f6c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008e3a0 0x7f76600bab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f7664f6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f7664f6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76600b5730 con 0x7f76600cf460 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f7664f6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76600b55b0 con 0x7f766008da90 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f7664f6c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76600cfdb0 con 0x7f766008e3a0 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f765ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f765ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52558/0 (socket says 172.21.15.94:52558) 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.157+0000 7f765ed9d640 1 -- 172.21.15.94:0/512350138 learned_addr learned my addr 172.21.15.94:0/512350138 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765e59c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f766008da90 0x7f76600a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765dd9b640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008e3a0 0x7f76600bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765ed9d640 1 -- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008e3a0 msgr2=0x7f76600bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765ed9d640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008e3a0 0x7f76600bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765ed9d640 1 -- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f766008da90 msgr2=0x7f76600a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765ed9d640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f766008da90 0x7f76600a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765ed9d640 1 -- 172.21.15.94:0/512350138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766009fc50 con 0x7f76600cf460 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765dd9b640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008e3a0 0x7f76600bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f765ed9d640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cd320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f765802a250 tx=0x7f76580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658041020 con 0x7f76600cf460 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658030990 con 0x7f76600cf460 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76600bb060 con 0x7f76600cf460 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658037040 con 0x7f76600cf460 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.158+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76600a03a0 con 0x7f76600cf460 2024-04-25T16:27:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.159+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766008d8b0 con 0x7f76600cf460 2024-04-25T16:27:45.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.160+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7658044020 con 0x7f76600cf460 2024-04-25T16:27:45.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.160+0000 7f7656ffd640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f76400744e0 0x7f76400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.160+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f76580b5fb0 con 0x7f76600cf460 2024-04-25T16:27:45.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.160+0000 7f765e59c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f76400744e0 0x7f76400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.161+0000 7f765e59c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f76400744e0 0x7f76400769f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7660074bb0 tx=0x7f765000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:45.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.165+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f766008d8b0 con 0x7f76600cf460 2024-04-25T16:27:45.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.354+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f766009fe20 con 0x7f76600cf460 2024-04-25T16:27:45.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.354+0000 7f7656ffd640 1 -- 172.21.15.94:0/512350138 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f765804b030 con 0x7f76600cf460 2024-04-25T16:27:45.355 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f76400744e0 msgr2=0x7f76400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f76400744e0 0x7f76400769f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7660074bb0 tx=0x7f765000d040 comp rx=0 tx=0).stop 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 msgr2=0x7f76600cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cd320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f765802a250 tx=0x7f76580046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 shutdown_connections 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f76400744e0 0x7f76400769f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76600cf460 0x7f76600cd320 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f766008e3a0 0x7f76600bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 --2- 172.21.15.94:0/512350138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f766008da90 0x7f76600a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 >> 172.21.15.94:0/512350138 conn(0x7f76600b5a60 msgr2=0x7f76600d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:45.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 shutdown_connections 2024-04-25T16:27:45.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.356+0000 7f7664f6c640 1 -- 172.21.15.94:0/512350138 wait complete. 2024-04-25T16:27:45.368 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T16:27:45.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:45.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T16:27:45.395 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:27:45.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:45.395 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T16:27:45.450 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:45.450 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:45.505 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:27:45.506 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:45.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:45.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.707+0000 7f0cba982640 1 Processor -- start 2024-04-25T16:27:45.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.708+0000 7f0cba982640 1 -- start start 2024-04-25T16:27:45.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cba982640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0cb40d0910 0x7f0cb40d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cba982640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0cb40b43f0 0x7f0cb40b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cba982640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb40d1260 con 0x7f0cb4164fa0 2024-04-25T16:27:45.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cba982640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb408d8a0 con 0x7f0cb40b43f0 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cba982640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb408db80 con 0x7f0cb40d0910 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb3fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0cb4164fa0 0x7f0cb4160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44390/0 (socket says 172.21.15.94:44390) 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb3fff640 1 -- 172.21.15.94:0/4023879133 learned_addr learned my addr 172.21.15.94:0/4023879133 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb8f1f640 1 --2- 172.21.15.94:0/4023879133 >> v2:172.21.15.94:3300/0 conn(0x7f0cb40b43f0 0x7f0cb40b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb37fe640 1 --2- 172.21.15.94:0/4023879133 >> v2:172.21.15.138:3300/0 conn(0x7f0cb40d0910 0x7f0cb40d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb8f1f640 1 -- 172.21.15.94:0/4023879133 >> v2:172.21.15.138:3300/0 conn(0x7f0cb40d0910 msgr2=0x7f0cb40d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb8f1f640 1 --2- 172.21.15.94:0/4023879133 >> v2:172.21.15.138:3300/0 conn(0x7f0cb40d0910 0x7f0cb40d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb8f1f640 1 -- 172.21.15.94:0/4023879133 >> v1:172.21.15.94:6789/0 conn(0x7f0cb4164fa0 legacy=0x7f0cb4160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.709+0000 7f0cb8f1f640 1 -- 172.21.15.94:0/4023879133 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb408de60 con 0x7f0cb40b43f0 2024-04-25T16:27:45.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.710+0000 7f0cb8f1f640 1 --2- 172.21.15.94:0/4023879133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40b43f0 0x7f0cb40b4800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40d1760 tx=0x7f0cac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e1a451285095d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:45.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cb2ffd640 1 -- 172.21.15.94:0/4023879133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac007740 con 0x7f0cb40b43f0 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cb2ffd640 1 -- 172.21.15.94:0/4023879133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac0078e0 con 0x7f0cb40b43f0 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cb2ffd640 1 -- 172.21.15.94:0/4023879133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac007cb0 con 0x7f0cb40b43f0 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cba982640 1 -- 172.21.15.94:0/4023879133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40b43f0 msgr2=0x7f0cb40b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cba982640 1 --2- 172.21.15.94:0/4023879133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40b43f0 0x7f0cb40b4800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40d1760 tx=0x7f0cac02a680 comp rx=0 tx=0).stop 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cba982640 1 -- 172.21.15.94:0/4023879133 shutdown_connections 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cba982640 1 --2- 172.21.15.94:0/4023879133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40b43f0 0x7f0cb40b4800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cba982640 1 --2- 172.21.15.94:0/4023879133 >> v2:172.21.15.138:3300/0 conn(0x7f0cb40d0910 0x7f0cb40d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.711+0000 7f0cba982640 1 -- 172.21.15.94:0/4023879133 >> 172.21.15.94:0/4023879133 conn(0x7f0cb40d4090 msgr2=0x7f0cb40d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.712+0000 7f0cba982640 1 -- 172.21.15.94:0/4023879133 shutdown_connections 2024-04-25T16:27:45.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.712+0000 7f0cba982640 1 -- 172.21.15.94:0/4023879133 wait complete. 2024-04-25T16:27:45.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.713+0000 7f0cba982640 1 Processor -- start 2024-04-25T16:27:45.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.714+0000 7f0cba982640 1 -- start start 2024-04-25T16:27:45.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cba982640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cb40b43f0 0x7f0cb41f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cba982640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 0x7f0cb4161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cba982640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb4164fa0 0x7f0cb41615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cba982640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb40d2400 con 0x7f0cb40d0910 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cba982640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb408e420 con 0x7f0cb4164fa0 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cba982640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb41653b0 con 0x7f0cb40b43f0 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 0x7f0cb4161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 0x7f0cb4161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52578/0 (socket says 172.21.15.94:52578) 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 -- 172.21.15.94:0/2691066127 learned_addr learned my addr 172.21.15.94:0/2691066127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb3fff640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cb40b43f0 0x7f0cb41f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb8f1f640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb4164fa0 0x7f0cb41615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 -- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cb40b43f0 msgr2=0x7f0cb41f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cb40b43f0 0x7f0cb41f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 -- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb4164fa0 msgr2=0x7f0cb41615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb4164fa0 0x7f0cb41615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 -- 172.21.15.94:0/2691066127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41f7100 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.715+0000 7f0cb37fe640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 0x7f0cb4161060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800e870 tx=0x7f0ca800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.716+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8017070 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.716+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca800c940 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.716+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb40b0d00 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.716+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800cd30 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.716+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb40b3730 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.717+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb406a180 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.717+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ca801a070 con 0x7f0cb40d0910 2024-04-25T16:27:45.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.717+0000 7f0cb0ff9640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0ca00744b0 0x7f0ca00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:45.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.718+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0ca80954b0 con 0x7f0cb40d0910 2024-04-25T16:27:45.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.718+0000 7f0cb3fff640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0ca00744b0 0x7f0ca00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:45.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.718+0000 7f0cb3fff640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0ca00744b0 0x7f0ca00769c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40010e0 tx=0x7f0ca4007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:45.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.722+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ca805f1f0 con 0x7f0cb40d0910 2024-04-25T16:27:45.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.897+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cb415d800 con 0x7f0cb40d0910 2024-04-25T16:27:45.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.898+0000 7f0cb0ff9640 1 -- 172.21.15.94:0/2691066127 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18868 (secure 0 0 0) 0x7f0ca80634f0 con 0x7f0cb40d0910 2024-04-25T16:27:45.899 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:45.899 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:36.681885+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:45.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0ca00744b0 msgr2=0x7f0ca00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0ca00744b0 0x7f0ca00769c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40010e0 tx=0x7f0ca4007d00 comp rx=0 tx=0).stop 2024-04-25T16:27:45.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 msgr2=0x7f0cb4161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:45.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 0x7f0cb4161060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800e870 tx=0x7f0ca800ed30 comp rx=0 tx=0).stop 2024-04-25T16:27:45.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 shutdown_connections 2024-04-25T16:27:45.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0ca00744b0 0x7f0ca00769c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb4164fa0 0x7f0cb41615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb40d0910 0x7f0cb4161060 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 --2- 172.21.15.94:0/2691066127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cb40b43f0 0x7f0cb41f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:45.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.900+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 >> 172.21.15.94:0/2691066127 conn(0x7f0cb40d4090 msgr2=0x7f0cb4160910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:45.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.901+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 shutdown_connections 2024-04-25T16:27:45.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:45.901+0000 7f0cba982640 1 -- 172.21.15.94:0/2691066127 wait complete. 2024-04-25T16:27:45.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:45.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:46.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.022+0000 7f004cb4d640 1 Processor -- start 2024-04-25T16:27:46.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.023+0000 7f004cb4d640 1 -- start start 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004cb4d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f00480b97b0 0x7f00480d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004cb4d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00480d3640 0x7f00480cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004cb4d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00480cf6d0 con 0x7f00480cf170 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004cb4d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f00480cfc90 con 0x7f00480b97b0 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004cb4d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00480b9bc0 con 0x7f00480d3640 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f00480b97b0 0x7f00480d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f00480b97b0 0x7f00480d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52580/0 (socket says 172.21.15.94:52580) 2024-04-25T16:27:46.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 -- 172.21.15.94:0/187197806 learned_addr learned my addr 172.21.15.94:0/187197806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f0045d9b640 1 --2- 172.21.15.94:0/187197806 >> v2:172.21.15.138:3300/0 conn(0x7f00480d3640 0x7f00480cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 -- 172.21.15.94:0/187197806 >> v2:172.21.15.138:3300/0 conn(0x7f00480d3640 msgr2=0x7f00480cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 --2- 172.21.15.94:0/187197806 >> v2:172.21.15.138:3300/0 conn(0x7f00480d3640 0x7f00480cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 -- 172.21.15.94:0/187197806 >> v1:172.21.15.138:6789/0 conn(0x7f00480cf170 legacy=0x7f00480b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.024+0000 7f004659c640 1 -- 172.21.15.94:0/187197806 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00480cf9b0 con 0x7f00480b97b0 2024-04-25T16:27:46.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.025+0000 7f004659c640 1 --2- 172.21.15.94:0/187197806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f00480d3100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00480d20b0 tx=0x7f004002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8cf0661dc40e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.025+0000 7f004559a640 1 -- 172.21.15.94:0/187197806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040037070 con 0x7f00480b97b0 2024-04-25T16:27:46.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.025+0000 7f004559a640 1 -- 172.21.15.94:0/187197806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0040032440 con 0x7f00480b97b0 2024-04-25T16:27:46.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.025+0000 7f004559a640 1 -- 172.21.15.94:0/187197806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040032810 con 0x7f00480b97b0 2024-04-25T16:27:46.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 -- 172.21.15.94:0/187197806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 msgr2=0x7f00480d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 --2- 172.21.15.94:0/187197806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f00480d3100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00480d20b0 tx=0x7f004002a200 comp rx=0 tx=0).stop 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 -- 172.21.15.94:0/187197806 shutdown_connections 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 --2- 172.21.15.94:0/187197806 >> v2:172.21.15.138:3300/0 conn(0x7f00480d3640 0x7f00480cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 --2- 172.21.15.94:0/187197806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f00480d3100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 -- 172.21.15.94:0/187197806 >> 172.21.15.94:0/187197806 conn(0x7f00480d3fa0 msgr2=0x7f00480d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 -- 172.21.15.94:0/187197806 shutdown_connections 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.026+0000 7f004cb4d640 1 -- 172.21.15.94:0/187197806 wait complete. 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.028+0000 7f004cb4d640 1 Processor -- start 2024-04-25T16:27:46.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.028+0000 7f004cb4d640 1 -- start start 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004cb4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f004815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004cb4d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00480cf170 0x7f004806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004cb4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00480d3640 0x7f004806f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004cb4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0048162360 con 0x7f00480b97b0 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004cb4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00480d2be0 con 0x7f00480d3640 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004cb4d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00480b7b50 con 0x7f00480cf170 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f004815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f004815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52586/0 (socket says 172.21.15.94:52586) 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.029+0000 7f004659c640 1 -- 172.21.15.94:0/4258892560 learned_addr learned my addr 172.21.15.94:0/4258892560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f0046d9d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00480d3640 0x7f004806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f0045d9b640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00480cf170 0x7f004806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004659c640 1 -- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00480cf170 msgr2=0x7f004806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004659c640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00480cf170 0x7f004806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004659c640 1 -- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00480d3640 msgr2=0x7f004806f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004659c640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00480d3640 0x7f004806f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004659c640 1 -- 172.21.15.94:0/4258892560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00480cbf90 con 0x7f00480b97b0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004659c640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f004815e450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0040000c00 tx=0x7f00400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040041020 con 0x7f00480b97b0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00400306c0 con 0x7f00480b97b0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00480cc230 con 0x7f00480b97b0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040037040 con 0x7f00480b97b0 2024-04-25T16:27:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.030+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004806e440 con 0x7f00480b97b0 2024-04-25T16:27:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.031+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004806a180 con 0x7f00480b97b0 2024-04-25T16:27:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.032+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0040044020 con 0x7f00480b97b0 2024-04-25T16:27:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.032+0000 7f0036ffd640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f00280744b0 0x7f00280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.032+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f00400b5a40 con 0x7f00480b97b0 2024-04-25T16:27:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.032+0000 7f0045d9b640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f00280744b0 0x7f00280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.033+0000 7f0045d9b640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f00280744b0 0x7f00280769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00480b5370 tx=0x7f003c0039b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.037+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f004004b070 con 0x7f00480b97b0 2024-04-25T16:27:46.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.211+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f00480d43b0 con 0x7f00480b97b0 2024-04-25T16:27:46.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.212+0000 7f0036ffd640 1 -- 172.21.15.94:0/4258892560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f004007fe60 con 0x7f00480b97b0 2024-04-25T16:27:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f00280744b0 msgr2=0x7f00280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f00280744b0 0x7f00280769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00480b5370 tx=0x7f003c0039b0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 msgr2=0x7f004815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f004815e450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0040000c00 tx=0x7f00400046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 shutdown_connections 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f00280744b0 0x7f00280769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00480d3640 0x7f004806f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00480cf170 0x7f004806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 --2- 172.21.15.94:0/4258892560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00480b97b0 0x7f004815e450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 >> 172.21.15.94:0/4258892560 conn(0x7f00480d3fa0 msgr2=0x7f004815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 shutdown_connections 2024-04-25T16:27:46.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.214+0000 7f004cb4d640 1 -- 172.21.15.94:0/4258892560 wait complete. 2024-04-25T16:27:46.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.335+0000 7f8e42735640 1 Processor -- start 2024-04-25T16:27:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.335+0000 7f8e42735640 1 -- start start 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e42735640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c164e40 0x7f8e3c165250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e42735640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8e3c165790 0x7f8e3c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e42735640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3c0d39e0 con 0x7f8e3c0b9780 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e42735640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8e3c0d2dd0 con 0x7f8e3c165790 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e42735640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c0d30b0 con 0x7f8e3c164e40 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e40cd2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8e3c165790 0x7f8e3c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e40cd2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8e3c165790 0x7f8e3c0ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52602/0 (socket says 172.21.15.94:52602) 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e40cd2640 1 -- 172.21.15.94:0/3902749777 learned_addr learned my addr 172.21.15.94:0/3902749777 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.336+0000 7f8e3b7fe640 1 --2- 172.21.15.94:0/3902749777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c164e40 0x7f8e3c165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.337+0000 7f8e40cd2640 1 -- 172.21.15.94:0/3902749777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c164e40 msgr2=0x7f8e3c165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.337+0000 7f8e40cd2640 1 --2- 172.21.15.94:0/3902749777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c164e40 0x7f8e3c165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.337+0000 7f8e40cd2640 1 -- 172.21.15.94:0/3902749777 >> v1:172.21.15.138:6789/0 conn(0x7f8e3c0b9780 legacy=0x7f8e3c09c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.337+0000 7f8e40cd2640 1 -- 172.21.15.94:0/3902749777 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c0d3390 con 0x7f8e3c165790 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.337+0000 7f8e40cd2640 1 --2- 172.21.15.94:0/3902749777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c165790 0x7f8e3c0ce960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0d3bb0 tx=0x7f8e3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f62337e12c306b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.337+0000 7f8e3affd640 1 -- 172.21.15.94:0/3902749777 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34036070 con 0x7f8e3c165790 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e3affd640 1 -- 172.21.15.94:0/3902749777 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e34031600 con 0x7f8e3c165790 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e3affd640 1 -- 172.21.15.94:0/3902749777 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340319d0 con 0x7f8e3c165790 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 -- 172.21.15.94:0/3902749777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c165790 msgr2=0x7f8e3c0ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 --2- 172.21.15.94:0/3902749777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c165790 0x7f8e3c0ce960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0d3bb0 tx=0x7f8e3402a710 comp rx=0 tx=0).stop 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 -- 172.21.15.94:0/3902749777 shutdown_connections 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 --2- 172.21.15.94:0/3902749777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c165790 0x7f8e3c0ce960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 --2- 172.21.15.94:0/3902749777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c164e40 0x7f8e3c165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 -- 172.21.15.94:0/3902749777 >> 172.21.15.94:0/3902749777 conn(0x7f8e3c0b5b20 msgr2=0x7f8e3c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 -- 172.21.15.94:0/3902749777 shutdown_connections 2024-04-25T16:27:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.338+0000 7f8e42735640 1 -- 172.21.15.94:0/3902749777 wait complete. 2024-04-25T16:27:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.340+0000 7f8e42735640 1 Processor -- start 2024-04-25T16:27:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.340+0000 7f8e42735640 1 -- start start 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.341+0000 7f8e42735640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c0b9780 0x7f8e3c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.341+0000 7f8e42735640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 0x7f8e3c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.341+0000 7f8e42735640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e3c165790 0x7f8e3c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.341+0000 7f8e42735640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c165ba0 con 0x7f8e3c164e40 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.341+0000 7f8e42735640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c09c9c0 con 0x7f8e3c165790 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.341+0000 7f8e42735640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c0b4f60 con 0x7f8e3c0b9780 2024-04-25T16:27:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 0x7f8e3c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 0x7f8e3c0cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52612/0 (socket says 172.21.15.94:52612) 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 -- 172.21.15.94:0/3070607656 learned_addr learned my addr 172.21.15.94:0/3070607656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 -- 172.21.15.94:0/3070607656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c0b9780 msgr2=0x7f8e3c15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c0b9780 0x7f8e3c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 -- 172.21.15.94:0/3070607656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e3c165790 msgr2=0x7f8e3c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e3c165790 0x7f8e3c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 -- 172.21.15.94:0/3070607656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c0c8e70 con 0x7f8e3c164e40 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e3b7fe640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 0x7f8e3c0cfbb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000d5d0 tx=0x7f8e3000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e300125f0 con 0x7f8e3c164e40 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30019070 con 0x7f8e3c164e40 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c0c9150 con 0x7f8e3c164e40 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e300129f0 con 0x7f8e3c164e40 2024-04-25T16:27:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.342+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c15d160 con 0x7f8e3c164e40 2024-04-25T16:27:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.343+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c0d02e0 con 0x7f8e3c164e40 2024-04-25T16:27:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.344+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e30012b90 con 0x7f8e3c164e40 2024-04-25T16:27:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.344+0000 7f8e38ff9640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8e280744e0 0x7f8e280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.344+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8e30095920 con 0x7f8e3c164e40 2024-04-25T16:27:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.344+0000 7f8e3bfff640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8e280744e0 0x7f8e280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.345+0000 7f8e3bfff640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8e280744e0 0x7f8e280769f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c000f60 tx=0x7f8e2c0079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.349+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e3005fdd0 con 0x7f8e3c164e40 2024-04-25T16:27:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.524+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e3c06a180 con 0x7f8e3c164e40 2024-04-25T16:27:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.525+0000 7f8e38ff9640 1 -- 172.21.15.94:0/3070607656 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f8e3005f6d0 con 0x7f8e3c164e40 2024-04-25T16:27:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8e280744e0 msgr2=0x7f8e280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8e280744e0 0x7f8e280769f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c000f60 tx=0x7f8e2c0079d0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 msgr2=0x7f8e3c0cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 0x7f8e3c0cfbb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000d5d0 tx=0x7f8e3000da90 comp rx=0 tx=0).stop 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 shutdown_connections 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8e280744e0 0x7f8e280769f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e3c165790 0x7f8e3c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3c164e40 0x7f8e3c0cfbb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 --2- 172.21.15.94:0/3070607656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e3c0b9780 0x7f8e3c15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 >> 172.21.15.94:0/3070607656 conn(0x7f8e3c0b5b20 msgr2=0x7f8e3c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 shutdown_connections 2024-04-25T16:27:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.527+0000 7f8e42735640 1 -- 172.21.15.94:0/3070607656 wait complete. 2024-04-25T16:27:46.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.647+0000 7fb0fbf55640 1 Processor -- start 2024-04-25T16:27:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.648+0000 7fb0fbf55640 1 -- start start 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0fbf55640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 0x7fb0f41608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0fbf55640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0f40d4a20 0x7fb0f40d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0fbf55640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f4161930 con 0x7fb0f40b4c70 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0fbf55640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb0f4161c10 con 0x7fb0f40d4a20 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0fbf55640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f4164d10 con 0x7fb0f40b5710 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0f9cf1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 0x7fb0f41608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0fa4f2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0f40d4a20 0x7fb0f40d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0f9cf1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 0x7fb0f41608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51264/0 (socket says 172.21.15.94:51264) 2024-04-25T16:27:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.649+0000 7fb0f9cf1640 1 -- 172.21.15.94:0/1989785077 learned_addr learned my addr 172.21.15.94:0/1989785077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.650+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 msgr2=0x7fb0f41608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.650+0000 7fb0fa4f2640 1 --2- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 0x7fb0f41608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.650+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/1989785077 >> v1:172.21.15.138:6789/0 conn(0x7fb0f40b4c70 legacy=0x7fb0f4160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.650+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/1989785077 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f4164ff0 con 0x7fb0f40d4a20 2024-04-25T16:27:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.650+0000 7fb0f9cf1640 1 --2- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 0x7fb0f41608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.650+0000 7fb0fa4f2640 1 --2- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40d4e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097a0 tx=0x7fb0ec02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15144141753fb8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0f8cef640 1 -- 172.21.15.94:0/1989785077 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec037070 con 0x7fb0f40d4a20 2024-04-25T16:27:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0f8cef640 1 -- 172.21.15.94:0/1989785077 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec02adb0 con 0x7fb0f40d4a20 2024-04-25T16:27:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0f8cef640 1 -- 172.21.15.94:0/1989785077 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec032660 con 0x7fb0f40d4a20 2024-04-25T16:27:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 -- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40d4a20 msgr2=0x7fb0f40d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40d4e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097a0 tx=0x7fb0ec02a060 comp rx=0 tx=0).stop 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 -- 172.21.15.94:0/1989785077 shutdown_connections 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40d4e30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/1989785077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b5710 0x7fb0f41608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 -- 172.21.15.94:0/1989785077 >> 172.21.15.94:0/1989785077 conn(0x7fb0f40d6930 msgr2=0x7fb0f40b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.651+0000 7fb0fbf55640 1 -- 172.21.15.94:0/1989785077 shutdown_connections 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.652+0000 7fb0fbf55640 1 -- 172.21.15.94:0/1989785077 wait complete. 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.653+0000 7fb0fbf55640 1 Processor -- start 2024-04-25T16:27:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.653+0000 7fb0fbf55640 1 -- start start 2024-04-25T16:27:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.654+0000 7fb0fbf55640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b4c70 0x7fb0f4155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.654+0000 7fb0fbf55640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40b5710 0x7fb0f415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fbf55640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40ba790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fbf55640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4161530 con 0x7fb0f40d4a20 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fbf55640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40b51c0 con 0x7fb0f40b5710 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fbf55640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f41616b0 con 0x7fb0f40b4c70 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40ba790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52622/0 (socket says 172.21.15.94:52622) 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/3638359909 learned_addr learned my addr 172.21.15.94:0/3638359909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0f9cf1640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b4c70 0x7fb0f4155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0f94f0640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40b5710 0x7fb0f415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b4c70 msgr2=0x7fb0f4155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b4c70 0x7fb0f4155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40b5710 msgr2=0x7fb0f415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40b5710 0x7fb0f415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 -- 172.21.15.94:0/3638359909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f415b190 con 0x7fb0f40d4a20 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0f94f0640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40b5710 0x7fb0f415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fa4f2640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40ba790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec02a0a0 tx=0x7fb0ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec041030 con 0x7fb0f40d4a20 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec02abb0 con 0x7fb0f40d4a20 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec037040 con 0x7fb0f40d4a20 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f41fa4e0 con 0x7fb0f40d4a20 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.655+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f41faaa0 con 0x7fb0f40d4a20 2024-04-25T16:27:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.656+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f415af00 con 0x7fb0f40d4a20 2024-04-25T16:27:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.657+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb0ec044070 con 0x7fb0f40d4a20 2024-04-25T16:27:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.657+0000 7fb0ea7fc640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb0d80744b0 0x7fb0d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.657+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb0ec0b57e0 con 0x7fb0f40d4a20 2024-04-25T16:27:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.657+0000 7fb0f9cf1640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb0d80744b0 0x7fb0d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.658+0000 7fb0f9cf1640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb0d80744b0 0x7fb0d80769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40d2920 tx=0x7fb0f000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.662+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0ec02ad80 con 0x7fb0f40d4a20 2024-04-25T16:27:46.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.836+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb0f406a180 con 0x7fb0f40d4a20 2024-04-25T16:27:46.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.836+0000 7fb0ea7fc640 1 -- 172.21.15.94:0/3638359909 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fb0ec07fc00 con 0x7fb0f40d4a20 2024-04-25T16:27:46.837 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb0d80744b0 msgr2=0x7fb0d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb0d80744b0 0x7fb0d80769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40d2920 tx=0x7fb0f000b040 comp rx=0 tx=0).stop 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 msgr2=0x7fb0f40ba790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40ba790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec02a0a0 tx=0x7fb0ec0046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 shutdown_connections 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fb0d80744b0 0x7fb0d80769c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0f40d4a20 0x7fb0f40ba790 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f40b5710 0x7fb0f415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 --2- 172.21.15.94:0/3638359909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40b4c70 0x7fb0f4155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 >> 172.21.15.94:0/3638359909 conn(0x7fb0f40d6930 msgr2=0x7fb0f406ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 shutdown_connections 2024-04-25T16:27:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.838+0000 7fb0fbf55640 1 -- 172.21.15.94:0/3638359909 wait complete. 2024-04-25T16:27:46.850 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.959+0000 7f9e67682640 1 Processor -- start 2024-04-25T16:27:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.960+0000 7f9e67682640 1 -- start start 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e67682640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e600d18a0 0x7f9e6016b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e67682640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e6006cdc0 con 0x7f9e601650b0 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e67682640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e6006d0a0 con 0x7f9e600b5460 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e67682640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9e6006d380 con 0x7f9e600d18a0 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e6541e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9e601650b0 0x7f9e6006c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44406/0 (socket says 172.21.15.94:44406) 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e6541e640 1 -- 172.21.15.94:0/2556505075 learned_addr learned my addr 172.21.15.94:0/2556505075 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741886789 0 0) 0x7f9e6006cdc0 con 0x7f9e601650b0 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e4c003520 con 0x7f9e601650b0 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.961+0000 7f9e65c1f640 1 --2- 172.21.15.94:0/2556505075 >> v2:172.21.15.138:3300/0 conn(0x7f9e600d18a0 0x7f9e6016b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1823367918 0 0) 0x7f9e4c003520 con 0x7f9e601650b0 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 >> v2:172.21.15.138:3300/0 conn(0x7f9e600d18a0 msgr2=0x7f9e6016b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 --2- 172.21.15.94:0/2556505075 >> v2:172.21.15.138:3300/0 conn(0x7f9e600d18a0 0x7f9e6016b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 >> v1:172.21.15.138:6789/0 conn(0x7f9e600b5460 legacy=0x7f9e600d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6006d660 con 0x7f9e601650b0 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f9e5c0019f0 con 0x7f9e601650b0 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 >> v1:172.21.15.94:6789/0 conn(0x7f9e601650b0 legacy=0x7f9e6006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c0039b0 0x7f9e4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c0051c0 0x7f9e4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006d380 con 0x7f9e4c004800 2024-04-25T16:27:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c00c160 con 0x7f9e4c0051c0 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c00c360 con 0x7f9e4c0039b0 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e64c1d640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e6541e640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c0039b0 0x7f9e4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e65c1f640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c0051c0 0x7f9e4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e64c1d640 1 -- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c0039b0 msgr2=0x7f9e4c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e64c1d640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c0039b0 0x7f9e4c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e64c1d640 1 -- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c0051c0 msgr2=0x7f9e4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e64c1d640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c0051c0 0x7f9e4c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e64c1d640 1 -- 172.21.15.94:0/2556505075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c003520 con 0x7f9e4c004800 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.962+0000 7f9e65c1f640 1 -- 172.21.15.94:0/2556505075 reap_dead start 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.963+0000 7f9e64c1d640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e50001750 tx=0x7f9e4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f96cda28cd3a53d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.963+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50001370 con 0x7f9e4c004800 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.963+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e500052d0 con 0x7f9e4c004800 2024-04-25T16:27:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.963+0000 7f9e57fff640 1 -- 172.21.15.94:0/2556505075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5002abc0 con 0x7f9e4c004800 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 -- 172.21.15.94:0/2556505075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c004800 msgr2=0x7f9e4c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e50001750 tx=0x7f9e4c005830 comp rx=0 tx=0).stop 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 -- 172.21.15.94:0/2556505075 shutdown_connections 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 --2- 172.21.15.94:0/2556505075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 -- 172.21.15.94:0/2556505075 >> 172.21.15.94:0/2556505075 conn(0x7f9e600b7e90 msgr2=0x7f9e600b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 -- 172.21.15.94:0/2556505075 shutdown_connections 2024-04-25T16:27:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.964+0000 7f9e67682640 1 -- 172.21.15.94:0/2556505075 wait complete. 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.966+0000 7f9e67682640 1 Processor -- start 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.966+0000 7f9e67682640 1 -- start start 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e67682640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c004800 0x7f9e60155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e67682640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e6015bcb0 0x7f9e6015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e67682640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 0x7f9e600bb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e67682640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e600b5870 con 0x7f9e600bb440 2024-04-25T16:27:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e67682640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006de60 con 0x7f9e6015bcb0 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e67682640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6006c570 con 0x7f9e4c004800 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 0x7f9e600bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 0x7f9e600bb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52648/0 (socket says 172.21.15.94:52648) 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 -- 172.21.15.94:0/1238999375 learned_addr learned my addr 172.21.15.94:0/1238999375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e6541e640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c004800 0x7f9e60155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e64c1d640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e6015bcb0 0x7f9e6015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 -- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c004800 msgr2=0x7f9e60155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c004800 0x7f9e60155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 -- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e6015bcb0 msgr2=0x7f9e6015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e6015bcb0 0x7f9e6015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 -- 172.21.15.94:0/1238999375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600bbd90 con 0x7f9e600bb440 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.967+0000 7f9e65c1f640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 0x7f9e600bb850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e600b5bf0 tx=0x7f9e58011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.968+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004440 con 0x7f9e600bb440 2024-04-25T16:27:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.968+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5801b070 con 0x7f9e600bb440 2024-04-25T16:27:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.968+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58003850 con 0x7f9e600bb440 2024-04-25T16:27:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.968+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e60156920 con 0x7f9e600bb440 2024-04-25T16:27:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.968+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e60156ee0 con 0x7f9e600bb440 2024-04-25T16:27:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.969+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6006a180 con 0x7f9e600bb440 2024-04-25T16:27:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.970+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9e580039f0 con 0x7f9e600bb440 2024-04-25T16:27:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.970+0000 7f9e55ffb640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9e440744b0 0x7f9e440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.970+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9e58095910 con 0x7f9e600bb440 2024-04-25T16:27:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.970+0000 7f9e6541e640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9e440744b0 0x7f9e440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.971+0000 7f9e6541e640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9e440744b0 0x7f9e440769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001330 tx=0x7f9e5c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:46.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:46.975+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e600bbd90 con 0x7f9e600bb440 2024-04-25T16:27:47.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.149+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9e600d4730 con 0x7f9e600bb440 2024-04-25T16:27:47.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.149+0000 7f9e55ffb640 1 -- 172.21.15.94:0/1238999375 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9e5805fdc0 con 0x7f9e600bb440 2024-04-25T16:27:47.150 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:47.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.151+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9e440744b0 msgr2=0x7f9e440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.151+0000 7f9e67682640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9e440744b0 0x7f9e440769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001330 tx=0x7f9e5c006040 comp rx=0 tx=0).stop 2024-04-25T16:27:47.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.151+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 msgr2=0x7f9e600bb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.151+0000 7f9e67682640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 0x7f9e600bb850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e600b5bf0 tx=0x7f9e58011920 comp rx=0 tx=0).stop 2024-04-25T16:27:47.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 shutdown_connections 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9e440744b0 0x7f9e440769c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e600bb440 0x7f9e600bb850 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e6015bcb0 0x7f9e6015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 --2- 172.21.15.94:0/1238999375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e4c004800 0x7f9e60155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 >> 172.21.15.94:0/1238999375 conn(0x7f9e600b7e90 msgr2=0x7f9e60161fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 shutdown_connections 2024-04-25T16:27:47.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.152+0000 7f9e67682640 1 -- 172.21.15.94:0/1238999375 wait complete. 2024-04-25T16:27:47.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:27:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.271+0000 7f601afb4640 1 Processor -- start 2024-04-25T16:27:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.271+0000 7f601afb4640 1 -- start start 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f601afb4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f601408d280 0x7f60140d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f601afb4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60140d2df0 0x7f601409d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f601afb4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60140b9c50 con 0x7f60140d0270 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f601afb4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f60140d1f80 con 0x7f60140d2df0 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f601afb4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f60140b93f0 con 0x7f601408d280 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f6019551640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60140d2df0 0x7f601409d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f6019551640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60140d2df0 0x7f601409d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52662/0 (socket says 172.21.15.94:52662) 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.272+0000 7f6019551640 1 -- 172.21.15.94:0/2794484180 learned_addr learned my addr 172.21.15.94:0/2794484180 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6013fff640 1 --2- 172.21.15.94:0/2794484180 >> v2:172.21.15.138:3300/0 conn(0x7f601408d280 0x7f60140d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6019551640 1 -- 172.21.15.94:0/2794484180 >> v2:172.21.15.138:3300/0 conn(0x7f601408d280 msgr2=0x7f60140d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6019551640 1 --2- 172.21.15.94:0/2794484180 >> v2:172.21.15.138:3300/0 conn(0x7f601408d280 0x7f60140d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6019551640 1 -- 172.21.15.94:0/2794484180 >> v1:172.21.15.138:6789/0 conn(0x7f60140d0270 legacy=0x7f60140b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6019551640 1 -- 172.21.15.94:0/2794484180 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60140d1ca0 con 0x7f60140d2df0 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6013fff640 1 --2- 172.21.15.94:0/2794484180 >> v2:172.21.15.138:3300/0 conn(0x7f601408d280 0x7f60140d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.273+0000 7f6019551640 1 --2- 172.21.15.94:0/2794484180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409d1c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60140ce6a0 tx=0x7f600c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d923cdb716a18b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f60137fe640 1 -- 172.21.15.94:0/2794484180 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c036070 con 0x7f60140d2df0 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f60137fe640 1 -- 172.21.15.94:0/2794484180 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c031600 con 0x7f60140d2df0 2024-04-25T16:27:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f60137fe640 1 -- 172.21.15.94:0/2794484180 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0319d0 con 0x7f60140d2df0 2024-04-25T16:27:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 -- 172.21.15.94:0/2794484180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 msgr2=0x7f601409d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 --2- 172.21.15.94:0/2794484180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409d1c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60140ce6a0 tx=0x7f600c02a6f0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 -- 172.21.15.94:0/2794484180 shutdown_connections 2024-04-25T16:27:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 --2- 172.21.15.94:0/2794484180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409d1c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 --2- 172.21.15.94:0/2794484180 >> v2:172.21.15.138:3300/0 conn(0x7f601408d280 0x7f60140d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 -- 172.21.15.94:0/2794484180 >> 172.21.15.94:0/2794484180 conn(0x7f60140d0b50 msgr2=0x7f60140d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 -- 172.21.15.94:0/2794484180 shutdown_connections 2024-04-25T16:27:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.274+0000 7f601afb4640 1 -- 172.21.15.94:0/2794484180 wait complete. 2024-04-25T16:27:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.276+0000 7f601afb4640 1 Processor -- start 2024-04-25T16:27:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.276+0000 7f601afb4640 1 -- start start 2024-04-25T16:27:47.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.277+0000 7f601afb4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f601408d280 0x7f601414d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60140b4a50 0x7f601409c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60140d1880 con 0x7f60140d2df0 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60140b98e0 con 0x7f601408d280 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60140d1a00 con 0x7f60140b4a50 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52678/0 (socket says 172.21.15.94:52678) 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 -- 172.21.15.94:0/1854874826 learned_addr learned my addr 172.21.15.94:0/1854874826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 -- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60140b4a50 msgr2=0x7f601409c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6019551640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60140b4a50 0x7f601409c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6018d50640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f601408d280 0x7f601414d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60140b4a50 0x7f601409c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 -- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f601408d280 msgr2=0x7f601414d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f601408d280 0x7f601414d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 -- 172.21.15.94:0/1854874826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60140b1b80 con 0x7f60140d2df0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f6013fff640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409c1b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f600000d800 tx=0x7f600000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000004030 con 0x7f60140d2df0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60000041d0 con 0x7f60140d2df0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60140b3f70 con 0x7f60140d2df0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000096a0 con 0x7f60140d2df0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.278+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60140b4530 con 0x7f60140d2df0 2024-04-25T16:27:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.279+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60140b18f0 con 0x7f60140d2df0 2024-04-25T16:27:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.280+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6000009840 con 0x7f60140d2df0 2024-04-25T16:27:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.281+0000 7f60117fa640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5ff80744b0 0x7f5ff80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.281+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f60000958c0 con 0x7f60140d2df0 2024-04-25T16:27:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.281+0000 7f6018d50640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5ff80744b0 0x7f5ff80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.281+0000 7f6018d50640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5ff80744b0 0x7f5ff80769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60140752d0 tx=0x7f6008008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:47.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.285+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f600005fce0 con 0x7f60140d2df0 2024-04-25T16:27:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.473+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60140b91d0 con 0x7f60140d2df0 2024-04-25T16:27:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.474+0000 7f60117fa640 1 -- 172.21.15.94:0/1854874826 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f60140b91d0 con 0x7f60140d2df0 2024-04-25T16:27:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:27:47.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5ff80744b0 msgr2=0x7f5ff80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5ff80744b0 0x7f5ff80769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60140752d0 tx=0x7f6008008040 comp rx=0 tx=0).stop 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 msgr2=0x7f601409c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409c1b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f600000d800 tx=0x7f600000dcc0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 shutdown_connections 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5ff80744b0 0x7f5ff80769c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60140b4a50 0x7f601409c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60140d2df0 0x7f601409c1b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 --2- 172.21.15.94:0/1854874826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f601408d280 0x7f601414d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 >> 172.21.15.94:0/1854874826 conn(0x7f60140d0b50 msgr2=0x7f6014072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 shutdown_connections 2024-04-25T16:27:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.476+0000 7f601afb4640 1 -- 172.21.15.94:0/1854874826 wait complete. 2024-04-25T16:27:47.487 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T16:27:47.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:47.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.2 2024-04-25T16:27:47.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:27:47.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:47.515 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T16:27:47.570 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:27:47.570 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:47.589 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:27:47.589 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:47.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:47.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.754+0000 7f3bf5692640 1 Processor -- start 2024-04-25T16:27:47.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.755+0000 7f3bf5692640 1 -- start start 2024-04-25T16:27:47.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bf5692640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3bf0161ab0 0x7f3bf006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bf5692640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf00b1650 con 0x7f3bf015d3e0 2024-04-25T16:27:47.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bf5692640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf0160fd0 con 0x7f3bf00b6970 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bf5692640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf00b1410 con 0x7f3bf0161ab0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bee7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3bf0161ab0 0x7f3bf006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3beeffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3bf015d3e0 0x7f3bf00b0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44412/0 (socket says 172.21.15.94:44412) 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3beeffd640 1 -- 172.21.15.94:0/3570923400 learned_addr learned my addr 172.21.15.94:0/3570923400 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bee7fc640 1 -- 172.21.15.94:0/3570923400 >> v1:172.21.15.138:6789/0 conn(0x7f3bf00b6970 legacy=0x7f3bf00b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bee7fc640 1 -- 172.21.15.94:0/3570923400 >> v1:172.21.15.94:6789/0 conn(0x7f3bf015d3e0 legacy=0x7f3bf00b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.756+0000 7f3bee7fc640 1 -- 172.21.15.94:0/3570923400 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf01612b0 con 0x7f3bf0161ab0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.757+0000 7f3bee7fc640 1 --2- 172.21.15.94:0/3570923400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf0161ab0 0x7f3bf006d580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f719c2519be6330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.757+0000 7f3bedffb640 1 -- 172.21.15.94:0/3570923400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4037070 con 0x7f3bf0161ab0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.757+0000 7f3bedffb640 1 -- 172.21.15.94:0/3570923400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be4032440 con 0x7f3bf0161ab0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.757+0000 7f3bedffb640 1 -- 172.21.15.94:0/3570923400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4032810 con 0x7f3bf0161ab0 2024-04-25T16:27:47.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 -- 172.21.15.94:0/3570923400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf0161ab0 msgr2=0x7f3bf006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 --2- 172.21.15.94:0/3570923400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf0161ab0 0x7f3bf006d580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be402a1e0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 -- 172.21.15.94:0/3570923400 shutdown_connections 2024-04-25T16:27:47.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 --2- 172.21.15.94:0/3570923400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf0161ab0 0x7f3bf006d580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 -- 172.21.15.94:0/3570923400 >> 172.21.15.94:0/3570923400 conn(0x7f3bf00d23b0 msgr2=0x7f3bf0073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:47.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 -- 172.21.15.94:0/3570923400 shutdown_connections 2024-04-25T16:27:47.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.758+0000 7f3bf5692640 1 -- 172.21.15.94:0/3570923400 wait complete. 2024-04-25T16:27:47.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.760+0000 7f3bf5692640 1 Processor -- start 2024-04-25T16:27:47.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.760+0000 7f3bf5692640 1 -- start start 2024-04-25T16:27:47.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bf5692640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bf00b6970 0x7f3bf009eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bf5692640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 0x7f3bf014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bf5692640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf0161ab0 0x7f3bf015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bf5692640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf006ded0 con 0x7f3bf015d3e0 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bf5692640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf006dd50 con 0x7f3bf0161ab0 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bf5692640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf0161620 con 0x7f3bf00b6970 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 0x7f3bf014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 0x7f3bf014b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52696/0 (socket says 172.21.15.94:52696) 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bee7fc640 1 -- 172.21.15.94:0/2364590731 learned_addr learned my addr 172.21.15.94:0/2364590731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3beeffd640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bf00b6970 0x7f3bf009eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.761+0000 7f3bef7fe640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf0161ab0 0x7f3bf015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bee7fc640 1 -- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bf00b6970 msgr2=0x7f3bf009eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bee7fc640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bf00b6970 0x7f3bf009eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bee7fc640 1 -- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf0161ab0 msgr2=0x7f3bf015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bee7fc640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf0161ab0 0x7f3bf015bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bee7fc640 1 -- 172.21.15.94:0/2364590731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf014bee0 con 0x7f3bf015d3e0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3beeffd640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bf00b6970 0x7f3bf009eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bee7fc640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 0x7f3bf014b9a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3be40098c0 tx=0x7f3be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4041070 con 0x7f3bf015d3e0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be403c930 con 0x7f3bf015d3e0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4037040 con 0x7f3bf015d3e0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf0159570 con 0x7f3bf015d3e0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.762+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf0159b10 con 0x7f3bf015d3e0 2024-04-25T16:27:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.763+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf009f740 con 0x7f3bf015d3e0 2024-04-25T16:27:47.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.764+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3be4044070 con 0x7f3bf015d3e0 2024-04-25T16:27:47.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.764+0000 7f3bd77fe640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bdc0744b0 0x7f3bdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:47.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.764+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3be40b6180 con 0x7f3bf015d3e0 2024-04-25T16:27:47.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.764+0000 7f3beeffd640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bdc0744b0 0x7f3bdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:47.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.765+0000 7f3beeffd640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bdc0744b0 0x7f3bdc0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3be00012e0 tx=0x7f3be0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:47.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.769+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3be4080630 con 0x7f3bf015d3e0 2024-04-25T16:27:47.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.947+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bf006a180 con 0x7f3bf015d3e0 2024-04-25T16:27:47.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.948+0000 7f3bd77fe640 1 -- 172.21.15.94:0/2364590731 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18868 (secure 0 0 0) 0x7f3be407ff30 con 0x7f3bf015d3e0 2024-04-25T16:27:47.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:47.949 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:36.681885+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bdc0744b0 msgr2=0x7f3bdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bdc0744b0 0x7f3bdc0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3be00012e0 tx=0x7f3be0009040 comp rx=0 tx=0).stop 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 msgr2=0x7f3bf014b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 0x7f3bf014b9a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3be40098c0 tx=0x7f3be40046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 shutdown_connections 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bdc0744b0 0x7f3bdc0769c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf0161ab0 0x7f3bf015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bf015d3e0 0x7f3bf014b9a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 --2- 172.21.15.94:0/2364590731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bf00b6970 0x7f3bf009eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:47.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 >> 172.21.15.94:0/2364590731 conn(0x7f3bf00d23b0 msgr2=0x7f3bf015fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:47.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 shutdown_connections 2024-04-25T16:27:47.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:47.950+0000 7f3bf5692640 1 -- 172.21.15.94:0/2364590731 wait complete. 2024-04-25T16:27:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:47.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.073+0000 7f3dd1eaa640 1 Processor -- start 2024-04-25T16:27:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.073+0000 7f3dd1eaa640 1 -- start start 2024-04-25T16:27:48.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dd1eaa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b97b0 0x7f3dcc0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dd1eaa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3dcc0b7440 0x7f3dcc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dd1eaa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc0cf6d0 con 0x7f3dcc0d3640 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dd1eaa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc0cfc90 con 0x7f3dcc0b7440 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dd1eaa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0b9bc0 con 0x7f3dcc0b97b0 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dcbfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3dcc0b7440 0x7f3dcc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dcbfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3dcc0b7440 0x7f3dcc0b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52698/0 (socket says 172.21.15.94:52698) 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.074+0000 7f3dcbfff640 1 -- 172.21.15.94:0/1851279865 learned_addr learned my addr 172.21.15.94:0/1851279865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dcb7fe640 1 --2- 172.21.15.94:0/1851279865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b97b0 0x7f3dcc0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dcbfff640 1 -- 172.21.15.94:0/1851279865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b97b0 msgr2=0x7f3dcc0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dcbfff640 1 --2- 172.21.15.94:0/1851279865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b97b0 0x7f3dcc0d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dcbfff640 1 -- 172.21.15.94:0/1851279865 >> v1:172.21.15.138:6789/0 conn(0x7f3dcc0d3640 legacy=0x7f3dcc0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dcbfff640 1 -- 172.21.15.94:0/1851279865 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0cf9b0 con 0x7f3dcc0b7440 2024-04-25T16:27:48.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dcbfff640 1 --2- 172.21.15.94:0/1851279865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0b7440 0x7f3dcc0b7850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0b7210 tx=0x7f3dc402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d504e94a96cffa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dca7fc640 1 -- 172.21.15.94:0/1851279865 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4036070 con 0x7f3dcc0b7440 2024-04-25T16:27:48.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dca7fc640 1 -- 172.21.15.94:0/1851279865 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4031600 con 0x7f3dcc0b7440 2024-04-25T16:27:48.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.075+0000 7f3dca7fc640 1 -- 172.21.15.94:0/1851279865 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40319d0 con 0x7f3dcc0b7440 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/1851279865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0b7440 msgr2=0x7f3dcc0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/1851279865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0b7440 0x7f3dcc0b7850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0b7210 tx=0x7f3dc402a6c0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/1851279865 shutdown_connections 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/1851279865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0b7440 0x7f3dcc0b7850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/1851279865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b97b0 0x7f3dcc0d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/1851279865 >> 172.21.15.94:0/1851279865 conn(0x7f3dcc0d3fa0 msgr2=0x7f3dcc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/1851279865 shutdown_connections 2024-04-25T16:27:48.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.076+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/1851279865 wait complete. 2024-04-25T16:27:48.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.078+0000 7f3dd1eaa640 1 Processor -- start 2024-04-25T16:27:48.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.078+0000 7f3dd1eaa640 1 -- start start 2024-04-25T16:27:48.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.079+0000 7f3dd1eaa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b7440 0x7f3dcc14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.079+0000 7f3dd1eaa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dcc0b97b0 0x7f3dcc0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.079+0000 7f3dd1eaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 0x7f3dcc0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.079+0000 7f3dd1eaa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0b7d90 con 0x7f3dcc0d3640 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.079+0000 7f3dd1eaa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0d2c20 con 0x7f3dcc0b97b0 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.079+0000 7f3dd1eaa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc162360 con 0x7f3dcc0b7440 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 0x7f3dcc0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 0x7f3dcc0cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52704/0 (socket says 172.21.15.94:52704) 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 -- 172.21.15.94:0/3339002235 learned_addr learned my addr 172.21.15.94:0/3339002235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcaffd640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dcc0b97b0 0x7f3dcc0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcb7fe640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b7440 0x7f3dcc14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 -- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b7440 msgr2=0x7f3dcc14ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b7440 0x7f3dcc14ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 -- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dcc0b97b0 msgr2=0x7f3dcc0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dcc0b97b0 0x7f3dcc0cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 -- 172.21.15.94:0/3339002235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0b3c20 con 0x7f3dcc0d3640 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcb7fe640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b7440 0x7f3dcc14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcaffd640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dcc0b97b0 0x7f3dcc0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dcbfff640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 0x7f3dcc0cda00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009f40 tx=0x7f3dc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4036050 con 0x7f3dcc0d3640 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4042070 con 0x7f3dcc0d3640 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc0b3e90 con 0x7f3dcc0d3640 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4009510 con 0x7f3dcc0d3640 2024-04-25T16:27:48.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.080+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc0b4450 con 0x7f3dcc0d3640 2024-04-25T16:27:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.081+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc15bf00 con 0x7f3dcc0d3640 2024-04-25T16:27:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.082+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3dc4007bd0 con 0x7f3dcc0d3640 2024-04-25T16:27:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.082+0000 7f3db3fff640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3db80744e0 0x7f3db80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.083+0000 7f3dcb7fe640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3db80744e0 0x7f3db80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.083+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3dc40b6070 con 0x7f3dcc0d3640 2024-04-25T16:27:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.083+0000 7f3dcb7fe640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3db80744e0 0x7f3db80769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc006fd0 tx=0x7f3dbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.087+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3dc404b020 con 0x7f3dcc0d3640 2024-04-25T16:27:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.261+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3dcc06a180 con 0x7f3dcc0d3640 2024-04-25T16:27:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.262+0000 7f3db3fff640 1 -- 172.21.15.94:0/3339002235 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f3dc4080400 con 0x7f3dcc0d3640 2024-04-25T16:27:48.262 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.263+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3db80744e0 msgr2=0x7f3db80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.263+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3db80744e0 0x7f3db80769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc006fd0 tx=0x7f3dbc008040 comp rx=0 tx=0).stop 2024-04-25T16:27:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.263+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 msgr2=0x7f3dcc0cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.263+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 0x7f3dcc0cda00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009f40 tx=0x7f3dc4004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 shutdown_connections 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3db80744e0 0x7f3db80769f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dcc0d3640 0x7f3dcc0cda00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dcc0b97b0 0x7f3dcc0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 --2- 172.21.15.94:0/3339002235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dcc0b7440 0x7f3dcc14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 >> 172.21.15.94:0/3339002235 conn(0x7f3dcc0d3fa0 msgr2=0x7f3dcc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 shutdown_connections 2024-04-25T16:27:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.264+0000 7f3dd1eaa640 1 -- 172.21.15.94:0/3339002235 wait complete. 2024-04-25T16:27:48.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:48.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.385+0000 7fe217be3640 1 Processor -- start 2024-04-25T16:27:48.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.385+0000 7fe217be3640 1 -- start start 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe217be3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2100b9780 0x7fe21009c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe217be3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100d39e0 con 0x7fe21006ea50 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe217be3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100d2dd0 con 0x7fe21009c7f0 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe217be3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe2100d30b0 con 0x7fe2100b9780 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe216180640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe21006ea50 0x7fe2100ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe216180640 1 -- 172.21.15.94:0/3391573868 learned_addr learned my addr 172.21.15.94:0/3391573868 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21597f640 1 --2- 172.21.15.94:0/3391573868 >> v2:172.21.15.138:3300/0 conn(0x7fe2100b9780 0x7fe21009c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3351193045 0 0) 0x7fe2100d39e0 con 0x7fe21006ea50 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1fc003520 con 0x7fe21006ea50 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1104382554 0 0) 0x7fe1fc003520 con 0x7fe21006ea50 2024-04-25T16:27:48.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 >> v2:172.21.15.138:3300/0 conn(0x7fe2100b9780 msgr2=0x7fe21009c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 --2- 172.21.15.94:0/3391573868 >> v2:172.21.15.138:3300/0 conn(0x7fe2100b9780 0x7fe21009c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 >> v1:172.21.15.138:6789/0 conn(0x7fe21009c7f0 legacy=0x7fe21006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2100d3390 con 0x7fe21006ea50 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fe208001690 con 0x7fe21006ea50 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21597f640 1 --2- 172.21.15.94:0/3391573868 >> v2:172.21.15.138:3300/0 conn(0x7fe2100b9780 0x7fe21009c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 >> v1:172.21.15.94:6789/0 conn(0x7fe21006ea50 legacy=0x7fe2100ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc0039b0 0x7fe1fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1fc004800 0x7fe1fc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1fc0051c0 0x7fe1fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100d30b0 con 0x7fe1fc004800 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc00c160 con 0x7fe1fc0039b0 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.387+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe1fc00c360 con 0x7fe1fc0051c0 2024-04-25T16:27:48.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21517e640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1fc004800 0x7fe1fc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21597f640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc0039b0 0x7fe1fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe216180640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1fc0051c0 0x7fe1fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21517e640 1 -- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1fc0051c0 msgr2=0x7fe1fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21517e640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1fc0051c0 0x7fe1fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21517e640 1 -- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc0039b0 msgr2=0x7fe1fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21517e640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc0039b0 0x7fe1fc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe21517e640 1 -- 172.21.15.94:0/3391573868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc003520 con 0x7fe1fc004800 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe216180640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1fc0051c0 0x7fe1fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.388+0000 7fe216180640 1 -- 172.21.15.94:0/3391573868 reap_dead start 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe21517e640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1fc004800 0x7fe1fc004c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe200001750 tx=0x7fe1fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7d2522522ed74d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200001370 con 0x7fe1fc004800 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2000052d0 con 0x7fe1fc004800 2024-04-25T16:27:48.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe21497d640 1 -- 172.21.15.94:0/3391573868 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20002abc0 con 0x7fe1fc004800 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 -- 172.21.15.94:0/3391573868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1fc004800 msgr2=0x7fe1fc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1fc004800 0x7fe1fc004c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe200001750 tx=0x7fe1fc005830 comp rx=0 tx=0).stop 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 -- 172.21.15.94:0/3391573868 shutdown_connections 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 --2- 172.21.15.94:0/3391573868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1fc004800 0x7fe1fc004c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 -- 172.21.15.94:0/3391573868 >> 172.21.15.94:0/3391573868 conn(0x7fe2100b5b20 msgr2=0x7fe2100b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 -- 172.21.15.94:0/3391573868 shutdown_connections 2024-04-25T16:27:48.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.389+0000 7fe217be3640 1 -- 172.21.15.94:0/3391573868 wait complete. 2024-04-25T16:27:48.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.391+0000 7fe217be3640 1 Processor -- start 2024-04-25T16:27:48.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.391+0000 7fe217be3640 1 -- start start 2024-04-25T16:27:48.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.392+0000 7fe217be3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc004800 0x7fe2100cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.392+0000 7fe217be3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2101f7160 0x7fe2101592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.392+0000 7fe217be3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 0x7fe2100cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.392+0000 7fe217be3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100b50e0 con 0x7fe2100cd340 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.392+0000 7fe217be3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100b4f60 con 0x7fe1fc004800 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.392+0000 7fe217be3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe21009d040 con 0x7fe2101f7160 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 0x7fe2100cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 0x7fe2100cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52724/0 (socket says 172.21.15.94:52724) 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 -- 172.21.15.94:0/4271683930 learned_addr learned my addr 172.21.15.94:0/4271683930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe21597f640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc004800 0x7fe2100cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe21517e640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2101f7160 0x7fe2101592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 -- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2101f7160 msgr2=0x7fe2101592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2101f7160 0x7fe2101592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 -- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc004800 msgr2=0x7fe2100cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc004800 0x7fe2100cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 -- 172.21.15.94:0/4271683930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2100cdc90 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe21597f640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc004800 0x7fe2100cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe21517e640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2101f7160 0x7fe2101592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe216180640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 0x7fe2100cd750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe2100d3bb0 tx=0x7fe20800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208010510 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2080106b0 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2100cb190 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208010aa0 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.393+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2100cb8e0 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.394+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2100cb360 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.395+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe208010c40 con 0x7fe2100cd340 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.395+0000 7fe2067fc640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1f40744e0 0x7fe1f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.395+0000 7fe21597f640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1f40744e0 0x7fe1f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.395+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe208094bd0 con 0x7fe2100cd340 2024-04-25T16:27:48.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.396+0000 7fe21597f640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1f40744e0 0x7fe1f40769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2100d5130 tx=0x7fe20c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.400+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2100cb360 con 0x7fe2100cd340 2024-04-25T16:27:48.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.575+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe21006a180 con 0x7fe2100cd340 2024-04-25T16:27:48.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.576+0000 7fe2067fc640 1 -- 172.21.15.94:0/4271683930 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fe2100cdc90 con 0x7fe2100cd340 2024-04-25T16:27:48.578 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.577+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1f40744e0 msgr2=0x7fe1f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.577+0000 7fe217be3640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1f40744e0 0x7fe1f40769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2100d5130 tx=0x7fe20c00b040 comp rx=0 tx=0).stop 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.577+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 msgr2=0x7fe2100cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.577+0000 7fe217be3640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 0x7fe2100cd750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe2100d3bb0 tx=0x7fe20800e960 comp rx=0 tx=0).stop 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 shutdown_connections 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe1f40744e0 0x7fe1f40769f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2100cd340 0x7fe2100cd750 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2101f7160 0x7fe2101592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 --2- 172.21.15.94:0/4271683930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc004800 0x7fe2100cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 >> 172.21.15.94:0/4271683930 conn(0x7fe2100b5b20 msgr2=0x7fe2101644a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:48.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 shutdown_connections 2024-04-25T16:27:48.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.578+0000 7fe217be3640 1 -- 172.21.15.94:0/4271683930 wait complete. 2024-04-25T16:27:48.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:48.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.700+0000 7f34ac357640 1 Processor -- start 2024-04-25T16:27:48.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.700+0000 7f34ac357640 1 -- start start 2024-04-25T16:27:48.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34ac357640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f34a4157940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34ac357640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34a40d2ba0 0x7f34a40d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34ac357640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40d34f0 con 0x7f34a41619b0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34ac357640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34a40b2ce0 con 0x7f34a40d2ba0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34ac357640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34a40b0990 con 0x7f34a40b24b0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34a98f2640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f34a41619b0 0x7f34a40b3070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44420/0 (socket says 172.21.15.94:44420) 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34a98f2640 1 -- 172.21.15.94:0/3559253325 learned_addr learned my addr 172.21.15.94:0/3559253325 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459369854 0 0) 0x7f3494001110 con 0x7f34a41619b0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34aa0f3640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f34a4157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3490003520 con 0x7f34a41619b0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.702+0000 7f34aa8f4640 1 --2- 172.21.15.94:0/3559253325 >> v2:172.21.15.138:3300/0 conn(0x7f34a40d2ba0 0x7f34a40d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 887111621 0 0) 0x7f3490003520 con 0x7f34a41619b0 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 msgr2=0x7f34a4157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f34a4157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 >> v2:172.21.15.138:3300/0 conn(0x7f34a40d2ba0 msgr2=0x7f34a40d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 --2- 172.21.15.94:0/3559253325 >> v2:172.21.15.138:3300/0 conn(0x7f34a40d2ba0 0x7f34a40d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40b2a00 con 0x7f34a41619b0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f3494001840 con 0x7f34a41619b0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 >> v1:172.21.15.94:6789/0 conn(0x7f34a41619b0 legacy=0x7f34a40b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34900039b0 0x7f3490003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f3490004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f3490005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34a40b2ce0 con 0x7f3490004800 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34a40b0990 con 0x7f34900039b0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f349000b9c0 con 0x7f34a40b24b0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a98f2640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f3490004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34aa0f3640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34900039b0 0x7f3490003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34aa8f4640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f3490005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a98f2640 1 -- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 msgr2=0x7f3490005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a98f2640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f3490005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a98f2640 1 -- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34900039b0 msgr2=0x7f3490003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a98f2640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34900039b0 0x7f3490003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34a98f2640 1 -- 172.21.15.94:0/3559253325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3490003520 con 0x7f3490004800 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34aa0f3640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34900039b0 0x7f3490003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.703+0000 7f34aa8f4640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a40b24b0 0x7f3490005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.704+0000 7f34a98f2640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f3490004c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34a406fad0 tx=0x7f3490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925478f206512d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.704+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349400c0e0 con 0x7f3490004800 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.704+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3494034070 con 0x7f3490004800 2024-04-25T16:27:48.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.704+0000 7f34a90f1640 1 -- 172.21.15.94:0/3559253325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494005cf0 con 0x7f3490004800 2024-04-25T16:27:48.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34ac357640 1 -- 172.21.15.94:0/3559253325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 msgr2=0x7f3490004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34ac357640 1 --2- 172.21.15.94:0/3559253325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f3490004c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34a406fad0 tx=0x7f3490005830 comp rx=0 tx=0).stop 2024-04-25T16:27:48.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34aa8f4640 1 -- 172.21.15.94:0/3559253325 reap_dead start 2024-04-25T16:27:48.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34ac357640 1 -- 172.21.15.94:0/3559253325 shutdown_connections 2024-04-25T16:27:48.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34ac357640 1 -- 172.21.15.94:0/3559253325 >> 172.21.15.94:0/3559253325 conn(0x7f34a40b7cc0 msgr2=0x7f34a40b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:48.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34ac357640 1 -- 172.21.15.94:0/3559253325 shutdown_connections 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.705+0000 7f34ac357640 1 -- 172.21.15.94:0/3559253325 wait complete. 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.706+0000 7f34ac357640 1 Processor -- start 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.706+0000 7f34ac357640 1 -- start start 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f34a409fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a40a02d0 0x7f34a41f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a41f6d50 0x7f34a409e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4161dc0 con 0x7f3490004800 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34a406f620 con 0x7f34a40a02d0 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34a40b0f10 con 0x7f34a41f6d50 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f34a409fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f34a409fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52748/0 (socket says 172.21.15.94:52748) 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 -- 172.21.15.94:0/2836145490 learned_addr learned my addr 172.21.15.94:0/2836145490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34a98f2640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a40a02d0 0x7f34a41f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa8f4640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a41f6d50 0x7f34a409e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 -- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a41f6d50 msgr2=0x7f34a409e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a41f6d50 0x7f34a409e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 -- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a40a02d0 msgr2=0x7f34a41f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a40a02d0 0x7f34a41f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 -- 172.21.15.94:0/2836145490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a409d6c0 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34a98f2640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a40a02d0 0x7f34a41f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa8f4640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a41f6d50 0x7f34a409e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34aa0f3640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f34a409fd90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f34a0007e70 tx=0x7f34a00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a001a020 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0016470 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a409d9a0 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016840 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.708+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a409df60 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.709+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a406a180 con 0x7f3490004800 2024-04-25T16:27:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.710+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34a0005d10 con 0x7f3490004800 2024-04-25T16:27:48.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.710+0000 7f349affd640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34880744b0 0x7f34880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:48.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.710+0000 7f34a98f2640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34880744b0 0x7f34880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:48.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.710+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f34a00963a0 con 0x7f3490004800 2024-04-25T16:27:48.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.711+0000 7f34a98f2640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34880744b0 0x7f34880769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3494033000 tx=0x7f3494009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.715+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34a0024030 con 0x7f3490004800 2024-04-25T16:27:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.891+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34a406ca10 con 0x7f3490004800 2024-04-25T16:27:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.891+0000 7f349affd640 1 -- 172.21.15.94:0/2836145490 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f34a00607c0 con 0x7f3490004800 2024-04-25T16:27:48.892 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34880744b0 msgr2=0x7f34880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34880744b0 0x7f34880769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3494033000 tx=0x7f3494009030 comp rx=0 tx=0).stop 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 msgr2=0x7f34a409fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f34a409fd90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f34a0007e70 tx=0x7f34a00115a0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 shutdown_connections 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34880744b0 0x7f34880769c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34a41f6d50 0x7f34a409e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a40a02d0 0x7f34a41f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 --2- 172.21.15.94:0/2836145490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3490004800 0x7f34a409fd90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 >> 172.21.15.94:0/2836145490 conn(0x7f34a40b7cc0 msgr2=0x7f34a40b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 shutdown_connections 2024-04-25T16:27:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:48.893+0000 7f34ac357640 1 -- 172.21.15.94:0/2836145490 wait complete. 2024-04-25T16:27:48.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.014+0000 7f3e4a11f640 1 Processor -- start 2024-04-25T16:27:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.014+0000 7f3e4a11f640 1 -- start start 2024-04-25T16:27:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e4a11f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3e440cf7b0 0x7f3e4409fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e4a11f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e44070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e4a11f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e440707b0 con 0x7f3e4415d740 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e4a11f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3e4406d930 con 0x7f3e440cf7b0 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e4a11f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e440d17e0 con 0x7f3e440703a0 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e44070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e42ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3e440cf7b0 0x7f3e4409fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e437fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3e4415d740 0x7f3e440cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:53946/0 (socket says 172.21.15.94:53946) 2024-04-25T16:27:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.015+0000 7f3e437fe640 1 -- 172.21.15.94:0/1641119922 learned_addr learned my addr 172.21.15.94:0/1641119922 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:49.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e43fff640 1 -- 172.21.15.94:0/1641119922 >> v2:172.21.15.138:3300/0 conn(0x7f3e440cf7b0 msgr2=0x7f3e4409fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:49.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e43fff640 1 --2- 172.21.15.94:0/1641119922 >> v2:172.21.15.138:3300/0 conn(0x7f3e440cf7b0 0x7f3e4409fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e43fff640 1 -- 172.21.15.94:0/1641119922 >> v1:172.21.15.138:6789/0 conn(0x7f3e4415d740 legacy=0x7f3e440cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e43fff640 1 -- 172.21.15.94:0/1641119922 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e440d1ac0 con 0x7f3e440703a0 2024-04-25T16:27:49.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e427fc640 1 -- 172.21.15.94:0/1641119922 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665504204 0 0) 0x7f3e440707b0 con 0x7f3e4415d740 2024-04-25T16:27:49.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e42ffd640 1 --2- 172.21.15.94:0/1641119922 >> v2:172.21.15.138:3300/0 conn(0x7f3e440cf7b0 0x7f3e4409fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:49.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.016+0000 7f3e43fff640 1 --2- 172.21.15.94:0/1641119922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e44070fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009d90 tx=0x7f3e3c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ffa4d2e13dae676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e427fc640 1 -- 172.21.15.94:0/1641119922 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c007740 con 0x7f3e440703a0 2024-04-25T16:27:49.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e427fc640 1 -- 172.21.15.94:0/1641119922 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c0078e0 con 0x7f3e440703a0 2024-04-25T16:27:49.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e427fc640 1 -- 172.21.15.94:0/1641119922 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c007cb0 con 0x7f3e440703a0 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 -- 172.21.15.94:0/1641119922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 msgr2=0x7f3e44070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/1641119922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e44070fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009d90 tx=0x7f3e3c02a660 comp rx=0 tx=0).stop 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 -- 172.21.15.94:0/1641119922 shutdown_connections 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/1641119922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e44070fa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/1641119922 >> v2:172.21.15.138:3300/0 conn(0x7f3e440cf7b0 0x7f3e4409fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 -- 172.21.15.94:0/1641119922 >> 172.21.15.94:0/1641119922 conn(0x7f3e440b4270 msgr2=0x7f3e440ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 -- 172.21.15.94:0/1641119922 shutdown_connections 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.017+0000 7f3e4a11f640 1 -- 172.21.15.94:0/1641119922 wait complete. 2024-04-25T16:27:49.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.019+0000 7f3e4a11f640 1 Processor -- start 2024-04-25T16:27:49.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.019+0000 7f3e4a11f640 1 -- start start 2024-04-25T16:27:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e4a11f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e4415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e4a11f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4415d740 0x7f3e4414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e4a11f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 0x7f3e440aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e4a11f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e440b50c0 con 0x7f3e4415db50 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e4a11f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e440b4f40 con 0x7f3e4415d740 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e4a11f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e440b5240 con 0x7f3e440703a0 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 0x7f3e440aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 0x7f3e440aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52756/0 (socket says 172.21.15.94:52756) 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 -- 172.21.15.94:0/3962157343 learned_addr learned my addr 172.21.15.94:0/3962157343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e42ffd640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4415d740 0x7f3e4414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e437fe640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e4415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 -- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 msgr2=0x7f3e4415b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e4415b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 -- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4415d740 msgr2=0x7f3e4414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4415d740 0x7f3e4414d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 -- 172.21.15.94:0/3962157343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441fa0e0 con 0x7f3e4415db50 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.021+0000 7f3e43fff640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 0x7f3e440aef20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009d90 tx=0x7f3e3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.022+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c03f070 con 0x7f3e4415db50 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.022+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c031c40 con 0x7f3e4415db50 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.022+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441fa3c0 con 0x7f3e4415db50 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.022+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c009450 con 0x7f3e4415db50 2024-04-25T16:27:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.022+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441fa980 con 0x7f3e4415db50 2024-04-25T16:27:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.023+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e441f9e50 con 0x7f3e4415db50 2024-04-25T16:27:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.023+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e3c003990 con 0x7f3e4415db50 2024-04-25T16:27:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.024+0000 7f3e2bfff640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3e300744b0 0x7f3e300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.024+0000 7f3e437fe640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3e300744b0 0x7f3e300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.024+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3e3c0b66d0 con 0x7f3e4415db50 2024-04-25T16:27:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.024+0000 7f3e437fe640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3e300744b0 0x7f3e300769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e34000f60 tx=0x7f3e34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.028+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e3c080410 con 0x7f3e4415db50 2024-04-25T16:27:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.201+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3e44158420 con 0x7f3e4415db50 2024-04-25T16:27:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.201+0000 7f3e2bfff640 1 -- 172.21.15.94:0/3962157343 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f3e3c084710 con 0x7f3e4415db50 2024-04-25T16:27:49.202 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3e300744b0 msgr2=0x7f3e300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3e300744b0 0x7f3e300769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e34000f60 tx=0x7f3e34009040 comp rx=0 tx=0).stop 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 msgr2=0x7f3e440aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 0x7f3e440aef20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009d90 tx=0x7f3e3c004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 shutdown_connections 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3e300744b0 0x7f3e300769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e4415db50 0x7f3e440aef20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4415d740 0x7f3e4414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 --2- 172.21.15.94:0/3962157343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e440703a0 0x7f3e4415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 >> 172.21.15.94:0/3962157343 conn(0x7f3e440b4270 msgr2=0x7f3e4406c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 shutdown_connections 2024-04-25T16:27:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.203+0000 7f3e4a11f640 1 -- 172.21.15.94:0/3962157343 wait complete. 2024-04-25T16:27:49.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:27:49.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.323+0000 7f5d9d500640 1 Processor -- start 2024-04-25T16:27:49.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.323+0000 7f5d9d500640 1 -- start start 2024-04-25T16:27:49.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d9d500640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d980d4ae0 0x7f5d98160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d9d500640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d981616d0 0x7f5d980d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d9d500640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d980d1770 con 0x7f5d980d1360 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d9d500640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5d980d50c0 con 0x7f5d981616d0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d9d500640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5d980d53a0 con 0x7f5d980d4ae0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d981616d0 0x7f5d980d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d981616d0 0x7f5d980d0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52762/0 (socket says 172.21.15.94:52762) 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 -- 172.21.15.94:0/1865441348 learned_addr learned my addr 172.21.15.94:0/1865441348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 -- 172.21.15.94:0/1865441348 >> v2:172.21.15.138:3300/0 conn(0x7f5d980d4ae0 msgr2=0x7f5d98160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d96ffd640 1 --2- 172.21.15.94:0/1865441348 >> v2:172.21.15.138:3300/0 conn(0x7f5d980d4ae0 0x7f5d98160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 --2- 172.21.15.94:0/1865441348 >> v2:172.21.15.138:3300/0 conn(0x7f5d980d4ae0 0x7f5d98160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 -- 172.21.15.94:0/1865441348 >> v1:172.21.15.138:6789/0 conn(0x7f5d980d1360 legacy=0x7f5d98164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.325+0000 7f5d967fc640 1 -- 172.21.15.94:0/1865441348 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d980d5680 con 0x7f5d981616d0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.326+0000 7f5d967fc640 1 --2- 172.21.15.94:0/1865441348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980d0e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5d9808e7b0 tx=0x7f5d8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9412ae5edca367e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.326+0000 7f5d95ffb640 1 -- 172.21.15.94:0/1865441348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c037070 con 0x7f5d981616d0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.326+0000 7f5d95ffb640 1 -- 172.21.15.94:0/1865441348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d8c032440 con 0x7f5d981616d0 2024-04-25T16:27:49.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.326+0000 7f5d95ffb640 1 -- 172.21.15.94:0/1865441348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c032810 con 0x7f5d981616d0 2024-04-25T16:27:49.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 -- 172.21.15.94:0/1865441348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 msgr2=0x7f5d980d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1865441348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980d0e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5d9808e7b0 tx=0x7f5d8c02a230 comp rx=0 tx=0).stop 2024-04-25T16:27:49.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 -- 172.21.15.94:0/1865441348 shutdown_connections 2024-04-25T16:27:49.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1865441348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980d0e20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1865441348 >> v2:172.21.15.138:3300/0 conn(0x7f5d980d4ae0 0x7f5d98160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 -- 172.21.15.94:0/1865441348 >> 172.21.15.94:0/1865441348 conn(0x7f5d980d6920 msgr2=0x7f5d980b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 -- 172.21.15.94:0/1865441348 shutdown_connections 2024-04-25T16:27:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.327+0000 7f5d9d500640 1 -- 172.21.15.94:0/1865441348 wait complete. 2024-04-25T16:27:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.329+0000 7f5d9d500640 1 Processor -- start 2024-04-25T16:27:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.329+0000 7f5d9d500640 1 -- start start 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d9d500640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d980d1360 0x7f5d981eae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d9d500640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d980d4ae0 0x7f5d980ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d9d500640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d9d500640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98161c60 con 0x7f5d981616d0 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d9d500640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98161ae0 con 0x7f5d980d4ae0 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d9d500640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9806d960 con 0x7f5d980d1360 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52768/0 (socket says 172.21.15.94:52768) 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 -- 172.21.15.94:0/1030402509 learned_addr learned my addr 172.21.15.94:0/1030402509 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d967fc640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d980d4ae0 0x7f5d980ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d96ffd640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d980d1360 0x7f5d981eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 -- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d980d1360 msgr2=0x7f5d981eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d980d1360 0x7f5d981eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 -- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d980d4ae0 msgr2=0x7f5d980ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d980d4ae0 0x7f5d980ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.330+0000 7f5d977fe640 1 -- 172.21.15.94:0/1030402509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d981ea0c0 con 0x7f5d981616d0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.331+0000 7f5d977fe640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980cd090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b410 tx=0x7f5d9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.331+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90015020 con 0x7f5d981616d0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.331+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d90011830 con 0x7f5d981616d0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.331+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d981ea3a0 con 0x7f5d981616d0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.331+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011c20 con 0x7f5d981616d0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.331+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d980cdaf0 con 0x7f5d981616d0 2024-04-25T16:27:49.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.332+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d981eb3c0 con 0x7f5d981616d0 2024-04-25T16:27:49.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.332+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d90018020 con 0x7f5d981616d0 2024-04-25T16:27:49.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.333+0000 7f5d7f7fe640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d840744b0 0x7f5d840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.333+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5d90095300 con 0x7f5d981616d0 2024-04-25T16:27:49.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.333+0000 7f5d96ffd640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d840744b0 0x7f5d840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.333+0000 7f5d96ffd640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d840744b0 0x7f5d840769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d980b84a0 tx=0x7f5d88007730 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.337+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d9001f020 con 0x7f5d981616d0 2024-04-25T16:27:49.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.524+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d980cdd50 con 0x7f5d981616d0 2024-04-25T16:27:49.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.524+0000 7f5d7f7fe640 1 -- 172.21.15.94:0/1030402509 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5d9005f0d0 con 0x7f5d981616d0 2024-04-25T16:27:49.525 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:27:49.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d840744b0 msgr2=0x7f5d840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d840744b0 0x7f5d840769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d980b84a0 tx=0x7f5d88007730 comp rx=0 tx=0).stop 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 msgr2=0x7f5d980cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980cd090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b410 tx=0x7f5d9000b8d0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 shutdown_connections 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f5d840744b0 0x7f5d840769c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d981616d0 0x7f5d980cd090 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d980d4ae0 0x7f5d980ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 --2- 172.21.15.94:0/1030402509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d980d1360 0x7f5d981eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.526+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 >> 172.21.15.94:0/1030402509 conn(0x7f5d980d6920 msgr2=0x7f5d980ba8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.527+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 shutdown_connections 2024-04-25T16:27:49.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.527+0000 7f5d9d500640 1 -- 172.21.15.94:0/1030402509 wait complete. 2024-04-25T16:27:49.538 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T16:27:49.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:49.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T16:27:49.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:27:49.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:49.571 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T16:27:49.638 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:27:49.639 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:49.696 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:27:49.696 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:27:49.754 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:49.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.861+0000 7f573cb7d640 1 Processor -- start 2024-04-25T16:27:49.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.861+0000 7f573cb7d640 1 -- start start 2024-04-25T16:27:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.862+0000 7f573cb7d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57380b5230 0x7f5738160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.862+0000 7f573cb7d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5738164d30 0x7f573806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.862+0000 7f573cb7d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57380b5a30 con 0x7f5738164210 2024-04-25T16:27:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.862+0000 7f573cb7d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f573806ca10 con 0x7f5738164d30 2024-04-25T16:27:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f573cb7d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57380d5610 con 0x7f57380b5230 2024-04-25T16:27:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f5736d9d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5738164d30 0x7f573806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f573659c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57380b5230 0x7f5738160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f5735d9b640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5738164210 0x7f5738164620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:53964/0 (socket says 172.21.15.94:53964) 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f5735d9b640 1 -- 172.21.15.94:0/1464430101 learned_addr learned my addr 172.21.15.94:0/1464430101 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f573659c640 1 -- 172.21.15.94:0/1464430101 >> v2:172.21.15.138:3300/0 conn(0x7f5738164d30 msgr2=0x7f573806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f573659c640 1 --2- 172.21.15.94:0/1464430101 >> v2:172.21.15.138:3300/0 conn(0x7f5738164d30 0x7f573806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f573659c640 1 -- 172.21.15.94:0/1464430101 >> v1:172.21.15.138:6789/0 conn(0x7f5738164210 legacy=0x7f5738164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f573659c640 1 -- 172.21.15.94:0/1464430101 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57380d58f0 con 0x7f57380b5230 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.863+0000 7f5736d9d640 1 --2- 172.21.15.94:0/1464430101 >> v2:172.21.15.138:3300/0 conn(0x7f5738164d30 0x7f573806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.864+0000 7f573659c640 1 --2- 172.21.15.94:0/1464430101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57380b5230 0x7f5738160790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f57280097a0 tx=0x7f572802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eeebdf09c359c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.864+0000 7f573559a640 1 -- 172.21.15.94:0/1464430101 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728037070 con 0x7f57380b5230 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.864+0000 7f573559a640 1 -- 172.21.15.94:0/1464430101 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572802ae50 con 0x7f57380b5230 2024-04-25T16:27:49.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.864+0000 7f573559a640 1 -- 172.21.15.94:0/1464430101 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728032660 con 0x7f57380b5230 2024-04-25T16:27:49.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 -- 172.21.15.94:0/1464430101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57380b5230 msgr2=0x7f5738160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 --2- 172.21.15.94:0/1464430101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57380b5230 0x7f5738160790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f57280097a0 tx=0x7f572802a100 comp rx=0 tx=0).stop 2024-04-25T16:27:49.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 -- 172.21.15.94:0/1464430101 shutdown_connections 2024-04-25T16:27:49.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 --2- 172.21.15.94:0/1464430101 >> v2:172.21.15.138:3300/0 conn(0x7f5738164d30 0x7f573806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 --2- 172.21.15.94:0/1464430101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57380b5230 0x7f5738160790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 -- 172.21.15.94:0/1464430101 >> 172.21.15.94:0/1464430101 conn(0x7f57380d3570 msgr2=0x7f57380b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:49.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 -- 172.21.15.94:0/1464430101 shutdown_connections 2024-04-25T16:27:49.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.865+0000 7f573cb7d640 1 -- 172.21.15.94:0/1464430101 wait complete. 2024-04-25T16:27:49.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.867+0000 7f573cb7d640 1 Processor -- start 2024-04-25T16:27:49.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.867+0000 7f573cb7d640 1 -- start start 2024-04-25T16:27:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f573cb7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57380b5230 0x7f57381fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f573cb7d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5738164210 0x7f573814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f573cb7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 0x7f57381f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f573cb7d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57380b9870 con 0x7f5738164d30 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f573cb7d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57380b96f0 con 0x7f57380b5230 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f573cb7d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f573806d3e0 con 0x7f5738164210 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f5736d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 0x7f57381f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f5736d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 0x7f57381f9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52772/0 (socket says 172.21.15.94:52772) 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.868+0000 7f5736d9d640 1 -- 172.21.15.94:0/3837552018 learned_addr learned my addr 172.21.15.94:0/3837552018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f573659c640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57380b5230 0x7f57381fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5735d9b640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5738164210 0x7f573814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5736d9d640 1 -- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5738164210 msgr2=0x7f573814e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5736d9d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5738164210 0x7f573814e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5736d9d640 1 -- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57380b5230 msgr2=0x7f57381fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5736d9d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57380b5230 0x7f57381fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5736d9d640 1 -- 172.21.15.94:0/3837552018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5738161840 con 0x7f5738164d30 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5736d9d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 0x7f57381f9c90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5730009e60 tx=0x7f573000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730018030 con 0x7f5738164d30 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573000cdf0 con 0x7f5738164d30 2024-04-25T16:27:49.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5738161b20 con 0x7f5738164d30 2024-04-25T16:27:49.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730003bb0 con 0x7f5738164d30 2024-04-25T16:27:49.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.869+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57381ea2d0 con 0x7f5738164d30 2024-04-25T16:27:49.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.870+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57381ec890 con 0x7f5738164d30 2024-04-25T16:27:49.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.871+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f573001b070 con 0x7f5738164d30 2024-04-25T16:27:49.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.871+0000 7f5726ffd640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57180744b0 0x7f57180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:49.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.871+0000 7f573659c640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57180744b0 0x7f57180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:49.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.871+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5730095f30 con 0x7f5738164d30 2024-04-25T16:27:49.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.872+0000 7f573659c640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57180744b0 0x7f57180769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5728034000 tx=0x7f572802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:49.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:49.876+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5730010040 con 0x7f5738164d30 2024-04-25T16:27:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.052+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f573806a180 con 0x7f5738164d30 2024-04-25T16:27:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.053+0000 7f5726ffd640 1 -- 172.21.15.94:0/3837552018 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18868 (secure 0 0 0) 0x7f5730060350 con 0x7f5738164d30 2024-04-25T16:27:50.053 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:50.054 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:36.681885+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57180744b0 msgr2=0x7f57180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57180744b0 0x7f57180769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5728034000 tx=0x7f572802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 msgr2=0x7f57381f9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 0x7f57381f9c90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5730009e60 tx=0x7f573000c490 comp rx=0 tx=0).stop 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 shutdown_connections 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f57180744b0 0x7f57180769c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5738164d30 0x7f57381f9c90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5738164210 0x7f573814e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 --2- 172.21.15.94:0/3837552018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57380b5230 0x7f57381fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 >> 172.21.15.94:0/3837552018 conn(0x7f57380d3570 msgr2=0x7f57380cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 shutdown_connections 2024-04-25T16:27:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.055+0000 7f573cb7d640 1 -- 172.21.15.94:0/3837552018 wait complete. 2024-04-25T16:27:50.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:26:43.312109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:26:53.603582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:26:54.860927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:26:58.564353+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:27:50.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:27:50.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.177+0000 7f47e6fa8640 1 Processor -- start 2024-04-25T16:27:50.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.177+0000 7f47e6fa8640 1 -- start start 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e6fa8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b8f90 0x7f47e00cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e6fa8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f47e00cf4f0 0x7f47e00b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e6fa8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e00d2db0 con 0x7f47e00b96b0 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e6fa8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f47e00b93a0 con 0x7f47e00cf4f0 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e6fa8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47e00d2b60 con 0x7f47e00b8f90 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e5545640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f47e00cf4f0 0x7f47e00b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e5545640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f47e00cf4f0 0x7f47e00b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52778/0 (socket says 172.21.15.94:52778) 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.178+0000 7f47e5545640 1 -- 172.21.15.94:0/788381334 learned_addr learned my addr 172.21.15.94:0/788381334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47dffff640 1 --2- 172.21.15.94:0/788381334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b8f90 0x7f47e00cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47e5545640 1 -- 172.21.15.94:0/788381334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b8f90 msgr2=0x7f47e00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47e5545640 1 --2- 172.21.15.94:0/788381334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b8f90 0x7f47e00cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47e5545640 1 -- 172.21.15.94:0/788381334 >> v1:172.21.15.138:6789/0 conn(0x7f47e00b96b0 legacy=0x7f47e00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47e5545640 1 -- 172.21.15.94:0/788381334 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e009d180 con 0x7f47e00cf4f0 2024-04-25T16:27:50.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47e5545640 1 --2- 172.21.15.94:0/788381334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e00b77e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009e50 tx=0x7f47d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5869931a94cc7672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.179+0000 7f47df7fe640 1 -- 172.21.15.94:0/788381334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8036070 con 0x7f47e00cf4f0 2024-04-25T16:27:50.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47df7fe640 1 -- 172.21.15.94:0/788381334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d8031600 con 0x7f47e00cf4f0 2024-04-25T16:27:50.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47df7fe640 1 -- 172.21.15.94:0/788381334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80319d0 con 0x7f47e00cf4f0 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 -- 172.21.15.94:0/788381334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 msgr2=0x7f47e00b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/788381334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e00b77e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009e50 tx=0x7f47d802a680 comp rx=0 tx=0).stop 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 -- 172.21.15.94:0/788381334 shutdown_connections 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/788381334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e00b77e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/788381334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b8f90 0x7f47e00cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 -- 172.21.15.94:0/788381334 >> 172.21.15.94:0/788381334 conn(0x7f47e00d0a00 msgr2=0x7f47e00d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 -- 172.21.15.94:0/788381334 shutdown_connections 2024-04-25T16:27:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.180+0000 7f47e6fa8640 1 -- 172.21.15.94:0/788381334 wait complete. 2024-04-25T16:27:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.182+0000 7f47e6fa8640 1 Processor -- start 2024-04-25T16:27:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.182+0000 7f47e6fa8640 1 -- start start 2024-04-25T16:27:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.183+0000 7f47e6fa8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e00b8f90 0x7f47e01e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.183+0000 7f47e6fa8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b96b0 0x7f47e01e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.183+0000 7f47e6fa8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e01e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.183+0000 7f47e6fa8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47e00d3090 con 0x7f47e00cf4f0 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.183+0000 7f47e6fa8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47e009d740 con 0x7f47e00b8f90 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.183+0000 7f47e6fa8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47e00b7d20 con 0x7f47e00b96b0 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e01e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e01e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52788/0 (socket says 172.21.15.94:52788) 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 -- 172.21.15.94:0/812628131 learned_addr learned my addr 172.21.15.94:0/812628131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47dffff640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b96b0 0x7f47e01e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e4d44640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e00b8f90 0x7f47e01e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 -- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b96b0 msgr2=0x7f47e01e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b96b0 0x7f47e01e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 -- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e00b8f90 msgr2=0x7f47e01e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e00b8f90 0x7f47e01e6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 -- 172.21.15.94:0/812628131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e01e88b0 con 0x7f47e00cf4f0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47dffff640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b96b0 0x7f47e01e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e4d44640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e00b8f90 0x7f47e01e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e5545640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e01e7b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009f70 tx=0x7f47d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d803f020 con 0x7f47e00cf4f0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d8037040 con 0x7f47e00cf4f0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e01e6110 con 0x7f47e00cf4f0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8036070 con 0x7f47e00cf4f0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.184+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e01e66d0 con 0x7f47e00cf4f0 2024-04-25T16:27:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.185+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e01e8620 con 0x7f47e00cf4f0 2024-04-25T16:27:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.186+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f47d8042020 con 0x7f47e00cf4f0 2024-04-25T16:27:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.186+0000 7f47dd7fa640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f47c40744b0 0x7f47c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.186+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f47d80b5bb0 con 0x7f47e00cf4f0 2024-04-25T16:27:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.186+0000 7f47e4d44640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f47c40744b0 0x7f47c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.187+0000 7f47e4d44640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f47c40744b0 0x7f47c40769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47d40015e0 tx=0x7f47d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.191+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47d8049030 con 0x7f47e00cf4f0 2024-04-25T16:27:50.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.367+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47e00d0e10 con 0x7f47e00cf4f0 2024-04-25T16:27:50.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.367+0000 7f47dd7fa640 1 -- 172.21.15.94:0/812628131 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f47e00d0e10 con 0x7f47e00cf4f0 2024-04-25T16:27:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:27:50.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f47c40744b0 msgr2=0x7f47c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f47c40744b0 0x7f47c40769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47d40015e0 tx=0x7f47d4009040 comp rx=0 tx=0).stop 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 msgr2=0x7f47e01e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e01e7b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009f70 tx=0x7f47d8004a40 comp rx=0 tx=0).stop 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 shutdown_connections 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f47c40744b0 0x7f47c40769c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e00cf4f0 0x7f47e01e7b80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47e00b96b0 0x7f47e01e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 --2- 172.21.15.94:0/812628131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e00b8f90 0x7f47e01e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 >> 172.21.15.94:0/812628131 conn(0x7f47e00d0a00 msgr2=0x7f47e00b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 shutdown_connections 2024-04-25T16:27:50.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.369+0000 7f47e6fa8640 1 -- 172.21.15.94:0/812628131 wait complete. 2024-04-25T16:27:50.381 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:27:50.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.492+0000 7f4f7eb72640 1 Processor -- start 2024-04-25T16:27:50.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.492+0000 7f4f7eb72640 1 -- start start 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7eb72640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4f7809bbc0 0x7f4f7806de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7eb72640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7806e770 con 0x7f4f7809bfd0 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7eb72640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f78161cf0 con 0x7f4f7806e360 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7eb72640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7815e8d0 con 0x7f4f7809bbc0 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f77fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4f7809bfd0 0x7f4f78160ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44422/0 (socket says 172.21.15.94:44422) 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f77fff640 1 -- 172.21.15.94:0/1191308977 learned_addr learned my addr 172.21.15.94:0/1191308977 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7c90e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4f7809bbc0 0x7f4f7806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7c90e640 1 -- 172.21.15.94:0/1191308977 >> v1:172.21.15.138:6789/0 conn(0x7f4f7806e360 legacy=0x7f4f781615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7c90e640 1 -- 172.21.15.94:0/1191308977 >> v1:172.21.15.94:6789/0 conn(0x7f4f7809bfd0 legacy=0x7f4f78160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f777fe640 1 -- 172.21.15.94:0/1191308977 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1793509646 0 0) 0x7f4f6c001110 con 0x7f4f7809bfd0 2024-04-25T16:27:50.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.493+0000 7f4f7c90e640 1 -- 172.21.15.94:0/1191308977 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f780b0db0 con 0x7f4f7809bbc0 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.494+0000 7f4f7c90e640 1 --2- 172.21.15.94:0/1191308977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bbc0 0x7f4f7806de20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4f680097a0 tx=0x7f4f6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b84b497dcd1e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.494+0000 7f4f777fe640 1 -- 172.21.15.94:0/1191308977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68037070 con 0x7f4f7809bbc0 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.494+0000 7f4f777fe640 1 -- 172.21.15.94:0/1191308977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68032440 con 0x7f4f7809bbc0 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.494+0000 7f4f777fe640 1 -- 172.21.15.94:0/1191308977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68032810 con 0x7f4f7809bbc0 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 -- 172.21.15.94:0/1191308977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bbc0 msgr2=0x7f4f7806de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/1191308977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bbc0 0x7f4f7806de20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4f680097a0 tx=0x7f4f6802a1e0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 -- 172.21.15.94:0/1191308977 shutdown_connections 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/1191308977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bbc0 0x7f4f7806de20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 -- 172.21.15.94:0/1191308977 >> 172.21.15.94:0/1191308977 conn(0x7f4f78074030 msgr2=0x7f4f780b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 -- 172.21.15.94:0/1191308977 shutdown_connections 2024-04-25T16:27:50.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.495+0000 7f4f7eb72640 1 -- 172.21.15.94:0/1191308977 wait complete. 2024-04-25T16:27:50.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.497+0000 7f4f7eb72640 1 Processor -- start 2024-04-25T16:27:50.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.497+0000 7f4f7eb72640 1 -- start start 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7eb72640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f7806e360 0x7f4f781530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7eb72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7809bbc0 0x7f4f78156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7eb72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 0x7f4f781573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7eb72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7806d580 con 0x7f4f7809bfd0 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7eb72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f780b19c0 con 0x7f4f7809bbc0 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7eb72640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4f780b1370 con 0x7f4f7806e360 2024-04-25T16:27:50.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7d10f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 0x7f4f781573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7d10f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 0x7f4f781573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52820/0 (socket says 172.21.15.94:52820) 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.498+0000 7f4f7d10f640 1 -- 172.21.15.94:0/2433323309 learned_addr learned my addr 172.21.15.94:0/2433323309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7c90e640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f7806e360 0x7f4f781530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f77fff640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7809bbc0 0x7f4f78156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7d10f640 1 -- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f7806e360 msgr2=0x7f4f781530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7d10f640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f7806e360 0x7f4f781530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7d10f640 1 -- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7809bbc0 msgr2=0x7f4f78156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7d10f640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7809bbc0 0x7f4f78156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7d10f640 1 -- 172.21.15.94:0/2433323309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f781555d0 con 0x7f4f7809bfd0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7c90e640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f7806e360 0x7f4f781530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f77fff640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7809bbc0 0x7f4f78156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7d10f640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 0x7f4f781573a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000ba10 tx=0x7f4f7000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70019070 con 0x7f4f7809bfd0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7000cb40 con 0x7f4f7809bfd0 2024-04-25T16:27:50.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f781558b0 con 0x7f4f7809bfd0 2024-04-25T16:27:50.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70014440 con 0x7f4f7809bfd0 2024-04-25T16:27:50.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.499+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f78155e70 con 0x7f4f7809bfd0 2024-04-25T16:27:50.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.500+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7806a180 con 0x7f4f7809bfd0 2024-04-25T16:27:50.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.501+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f7001c070 con 0x7f4f7809bfd0 2024-04-25T16:27:50.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.501+0000 7f4f757fa640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4f640744b0 0x7f4f640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.501+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4f70095260 con 0x7f4f7809bfd0 2024-04-25T16:27:50.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.501+0000 7f4f7c90e640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4f640744b0 0x7f4f640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.502+0000 7f4f7c90e640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4f640744b0 0x7f4f640769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f68034000 tx=0x7f4f6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.506+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f7005f5f0 con 0x7f4f7809bfd0 2024-04-25T16:27:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.679+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f7815e4a0 con 0x7f4f7809bfd0 2024-04-25T16:27:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.679+0000 7f4f757fa640 1 -- 172.21.15.94:0/2433323309 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f4f7005eef0 con 0x7f4f7809bfd0 2024-04-25T16:27:50.680 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4f640744b0 msgr2=0x7f4f640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4f640744b0 0x7f4f640769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f68034000 tx=0x7f4f6802e000 comp rx=0 tx=0).stop 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 msgr2=0x7f4f781573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 0x7f4f781573a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000ba10 tx=0x7f4f7000bed0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 shutdown_connections 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4f640744b0 0x7f4f640769c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f7809bfd0 0x7f4f781573a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7809bbc0 0x7f4f78156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 --2- 172.21.15.94:0/2433323309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f7806e360 0x7f4f781530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 >> 172.21.15.94:0/2433323309 conn(0x7f4f78074030 msgr2=0x7f4f7815fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 shutdown_connections 2024-04-25T16:27:50.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.681+0000 7f4f7eb72640 1 -- 172.21.15.94:0/2433323309 wait complete. 2024-04-25T16:27:50.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:27:50.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.804+0000 7fd9d6e92640 1 Processor -- start 2024-04-25T16:27:50.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.804+0000 7fd9d6e92640 1 -- start start 2024-04-25T16:27:50.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d6e92640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd9d00d3890 0x7fd9d00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d6e92640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9d00cf140 0x7fd9d00d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d6e92640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d006f840 con 0x7fd9d00cfcf0 2024-04-25T16:27:50.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d6e92640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d00d0830 con 0x7fd9d00cf140 2024-04-25T16:27:50.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d6e92640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d00b2810 con 0x7fd9d00d3890 2024-04-25T16:27:50.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d542f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9d00cf140 0x7fd9d00d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d542f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9d00cf140 0x7fd9d00d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52830/0 (socket says 172.21.15.94:52830) 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9cffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd9d00cfcf0 0x7fd9d00cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44424/0 (socket says 172.21.15.94:44424) 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.805+0000 7fd9d542f640 1 -- 172.21.15.94:0/2118452824 learned_addr learned my addr 172.21.15.94:0/2118452824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9d4c2e640 1 --2- 172.21.15.94:0/2118452824 >> v2:172.21.15.138:3300/0 conn(0x7fd9d00d3890 0x7fd9d00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9d542f640 1 -- 172.21.15.94:0/2118452824 >> v2:172.21.15.138:3300/0 conn(0x7fd9d00d3890 msgr2=0x7fd9d00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9d542f640 1 --2- 172.21.15.94:0/2118452824 >> v2:172.21.15.138:3300/0 conn(0x7fd9d00d3890 0x7fd9d00b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9d542f640 1 -- 172.21.15.94:0/2118452824 >> v1:172.21.15.94:6789/0 conn(0x7fd9d00cfcf0 legacy=0x7fd9d00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9d542f640 1 -- 172.21.15.94:0/2118452824 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d00b2af0 con 0x7fd9d00cf140 2024-04-25T16:27:50.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/2118452824 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622258952 0 0) 0x7fd9d006f840 con 0x7fd9d00cfcf0 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9d542f640 1 --2- 172.21.15.94:0/2118452824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d00d02f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00cf810 tx=0x7fd9c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260b82b5e3375349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.806+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/2118452824 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8036070 con 0x7fd9d00cf140 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/2118452824 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c8031600 con 0x7fd9d00cf140 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/2118452824 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80319d0 con 0x7fd9d00cf140 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2118452824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 msgr2=0x7fd9d00d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2118452824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d00d02f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00cf810 tx=0x7fd9c802a730 comp rx=0 tx=0).stop 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2118452824 shutdown_connections 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2118452824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d00d02f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2118452824 >> v2:172.21.15.138:3300/0 conn(0x7fd9d00d3890 0x7fd9d00b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2118452824 >> 172.21.15.94:0/2118452824 conn(0x7fd9d00b7b20 msgr2=0x7fd9d00b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2118452824 shutdown_connections 2024-04-25T16:27:50.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.807+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2118452824 wait complete. 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.809+0000 7fd9d6e92640 1 Processor -- start 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.809+0000 7fd9d6e92640 1 -- start start 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.810+0000 7fd9d6e92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.810+0000 7fd9d6e92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d00cfcf0 0x7fd9d0160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.810+0000 7fd9d6e92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d00d3890 0x7fd9d0160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.810+0000 7fd9d6e92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00b3390 con 0x7fd9d00cf140 2024-04-25T16:27:50.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.810+0000 7fd9d6e92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00b30b0 con 0x7fd9d00cfcf0 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.810+0000 7fd9d6e92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d00b3f80 con 0x7fd9d00d3890 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52840/0 (socket says 172.21.15.94:52840) 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 -- 172.21.15.94:0/2216603904 learned_addr learned my addr 172.21.15.94:0/2216603904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9cffff640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d00cfcf0 0x7fd9d0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d542f640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d00d3890 0x7fd9d0160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 -- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d00d3890 msgr2=0x7fd9d0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d00d3890 0x7fd9d0160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 -- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d00cfcf0 msgr2=0x7fd9d0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d00cfcf0 0x7fd9d0160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 -- 172.21.15.94:0/2216603904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d014ac00 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9cffff640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d00cfcf0 0x7fd9d0160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d4c2e640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d015ed30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c7f0 tx=0x7fd9c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4015070 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4004220 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d0161720 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4007620 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.811+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d0161400 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.812+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d006a180 con 0x7fd9d00cf140 2024-04-25T16:27:50.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.813+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9c40079d0 con 0x7fd9d00cf140 2024-04-25T16:27:50.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.813+0000 7fd9cd7fa640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd9b40744b0 0x7fd9b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:50.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.814+0000 7fd9cffff640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd9b40744b0 0x7fd9b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:50.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.814+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd9c4095630 con 0x7fd9d00cf140 2024-04-25T16:27:50.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.814+0000 7fd9cffff640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd9b40744b0 0x7fd9b40769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd9d014dc80 tx=0x7fd9bc007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:50.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.818+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9c405f9b0 con 0x7fd9d00cf140 2024-04-25T16:27:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.993+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9d0161c40 con 0x7fd9d00cf140 2024-04-25T16:27:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.994+0000 7fd9cd7fa640 1 -- 172.21.15.94:0/2216603904 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fd9c405f2b0 con 0x7fd9d00cf140 2024-04-25T16:27:50.995 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd9b40744b0 msgr2=0x7fd9b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd9b40744b0 0x7fd9b40769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd9d014dc80 tx=0x7fd9bc007cc0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 msgr2=0x7fd9d015ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d015ed30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c7f0 tx=0x7fd9c400ccb0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 shutdown_connections 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fd9b40744b0 0x7fd9b40769c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d00d3890 0x7fd9d0160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d00cfcf0 0x7fd9d0160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 --2- 172.21.15.94:0/2216603904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d00cf140 0x7fd9d015ed30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:50.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 >> 172.21.15.94:0/2216603904 conn(0x7fd9d00b7b20 msgr2=0x7fd9d00b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:50.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 shutdown_connections 2024-04-25T16:27:50.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:50.996+0000 7fd9d6e92640 1 -- 172.21.15.94:0/2216603904 wait complete. 2024-04-25T16:27:51.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:27:51.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.116+0000 7efd42d00640 1 Processor -- start 2024-04-25T16:27:51.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.116+0000 7efd42d00640 1 -- start start 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd42d00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efd3c0cf0a0 0x7efd3c0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd42d00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3c0cf9f0 con 0x7efd3c06d450 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd42d00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3c06df40 con 0x7efd3c06f7e0 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd42d00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7efd3c15ef90 con 0x7efd3c0cf0a0 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd3bfff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7efd3c06d450 0x7efd3c15f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44428/0 (socket says 172.21.15.94:44428) 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd3bfff640 1 -- 172.21.15.94:0/3864113477 learned_addr learned my addr 172.21.15.94:0/3864113477 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd4129d640 1 --2- 172.21.15.94:0/3864113477 >> v2:172.21.15.94:3300/0 conn(0x7efd3c0cf0a0 0x7efd3c0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd3b7fe640 1 -- 172.21.15.94:0/3864113477 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442948751 0 0) 0x7efd30001110 con 0x7efd3c06d450 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd3b7fe640 1 -- 172.21.15.94:0/3864113477 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd24003520 con 0x7efd3c06d450 2024-04-25T16:27:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd4129d640 1 -- 172.21.15.94:0/3864113477 >> v1:172.21.15.138:6789/0 conn(0x7efd3c06f7e0 legacy=0x7efd3c0d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd4129d640 1 -- 172.21.15.94:0/3864113477 >> v1:172.21.15.94:6789/0 conn(0x7efd3c06d450 legacy=0x7efd3c15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.118+0000 7efd4129d640 1 -- 172.21.15.94:0/3864113477 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c15ec40 con 0x7efd3c0cf0a0 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.119+0000 7efd4129d640 1 --2- 172.21.15.94:0/3864113477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c0cf4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efd34009d90 tx=0x7efd3402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b3e426d3c353f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.119+0000 7efd3b7fe640 1 -- 172.21.15.94:0/3864113477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34007740 con 0x7efd3c0cf0a0 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.119+0000 7efd3b7fe640 1 -- 172.21.15.94:0/3864113477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd340078e0 con 0x7efd3c0cf0a0 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.119+0000 7efd3b7fe640 1 -- 172.21.15.94:0/3864113477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34007cb0 con 0x7efd3c0cf0a0 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 -- 172.21.15.94:0/3864113477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c0cf0a0 msgr2=0x7efd3c0cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 --2- 172.21.15.94:0/3864113477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c0cf4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efd34009d90 tx=0x7efd3402a6f0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 -- 172.21.15.94:0/3864113477 shutdown_connections 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 --2- 172.21.15.94:0/3864113477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c0cf4b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 -- 172.21.15.94:0/3864113477 >> 172.21.15.94:0/3864113477 conn(0x7efd3c0b4270 msgr2=0x7efd3c0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 -- 172.21.15.94:0/3864113477 shutdown_connections 2024-04-25T16:27:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.120+0000 7efd42d00640 1 -- 172.21.15.94:0/3864113477 wait complete. 2024-04-25T16:27:51.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.122+0000 7efd42d00640 1 Processor -- start 2024-04-25T16:27:51.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.122+0000 7efd42d00640 1 -- start start 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd42d00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06d450 0x7efd3c1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd42d00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 0x7efd3c1ea200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd42d00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c09ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd42d00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c070040 con 0x7efd3c06f7e0 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd42d00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c06fd60 con 0x7efd3c0cf0a0 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd42d00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c070320 con 0x7efd3c06d450 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 0x7efd3c1ea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 0x7efd3c1ea200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52854/0 (socket says 172.21.15.94:52854) 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd3bfff640 1 -- 172.21.15.94:0/3644429995 learned_addr learned my addr 172.21.15.94:0/3644429995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.123+0000 7efd40a9c640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06d450 0x7efd3c1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd4129d640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c09ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd3bfff640 1 -- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06d450 msgr2=0x7efd3c1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd3bfff640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06d450 0x7efd3c1498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd3bfff640 1 -- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c0cf0a0 msgr2=0x7efd3c09ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd3bfff640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c09ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd3bfff640 1 -- 172.21.15.94:0/3644429995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c1470e0 con 0x7efd3c06f7e0 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd3bfff640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 0x7efd3c1ea200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c400 tx=0x7efd3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30017070 con 0x7efd3c06f7e0 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30005ce0 con 0x7efd3c06f7e0 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c09c9d0 con 0x7efd3c06f7e0 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300072d0 con 0x7efd3c06f7e0 2024-04-25T16:27:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.124+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c09cf90 con 0x7efd3c06f7e0 2024-04-25T16:27:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.125+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c06a180 con 0x7efd3c06f7e0 2024-04-25T16:27:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.126+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd3001a070 con 0x7efd3c06f7e0 2024-04-25T16:27:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.126+0000 7efd397fa640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efd280744b0 0x7efd280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.126+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efd30094930 con 0x7efd3c06f7e0 2024-04-25T16:27:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.126+0000 7efd40a9c640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efd280744b0 0x7efd280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.127+0000 7efd40a9c640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efd280744b0 0x7efd280769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007fd0 tx=0x7efd2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.131+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd3c0cf9f0 con 0x7efd3c06f7e0 2024-04-25T16:27:51.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.304+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efd3c09c760 con 0x7efd3c06f7e0 2024-04-25T16:27:51.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.305+0000 7efd397fa640 1 -- 172.21.15.94:0/3644429995 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7efd3005e670 con 0x7efd3c06f7e0 2024-04-25T16:27:51.305 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:27:51.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.306+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efd280744b0 msgr2=0x7efd280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.306+0000 7efd42d00640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efd280744b0 0x7efd280769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007fd0 tx=0x7efd2c009040 comp rx=0 tx=0).stop 2024-04-25T16:27:51.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.306+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 msgr2=0x7efd3c1ea200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.306+0000 7efd42d00640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 0x7efd3c1ea200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c400 tx=0x7efd3000c8c0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 shutdown_connections 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7efd280744b0 0x7efd280769c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c0cf0a0 0x7efd3c09ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3c06f7e0 0x7efd3c1ea200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 --2- 172.21.15.94:0/3644429995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06d450 0x7efd3c1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 >> 172.21.15.94:0/3644429995 conn(0x7efd3c0b4270 msgr2=0x7efd3c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 shutdown_connections 2024-04-25T16:27:51.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.307+0000 7efd42d00640 1 -- 172.21.15.94:0/3644429995 wait complete. 2024-04-25T16:27:51.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:27:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.426+0000 7f1409d38640 1 Processor -- start 2024-04-25T16:27:51.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.426+0000 7f1409d38640 1 -- start start 2024-04-25T16:27:51.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f1409d38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 0x7f14040d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f1409d38640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14040cdba0 con 0x7f14040b7c60 2024-04-25T16:27:51.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f1409d38640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14040cde40 con 0x7f14040d1160 2024-04-25T16:27:51.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f1409d38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f140415da40 con 0x7f140415d630 2024-04-25T16:27:51.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f1403fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f14040b7c60 0x7f14040b2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48000/0 (socket says 172.21.15.94:48000) 2024-04-25T16:27:51.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f1403fff640 1 -- 172.21.15.94:0/1520894107 learned_addr learned my addr 172.21.15.94:0/1520894107 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:51.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f14037fe640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 0x7f14040d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.427+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598808478 0 0) 0x7f14040cdba0 con 0x7f14040b7c60 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13ec003520 con 0x7f14040b7c60 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359082182 0 0) 0x7f14040cde40 con 0x7f14040d1160 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14040cdba0 con 0x7f14040d1160 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 14606202 0 0) 0x7f13ec003520 con 0x7f14040b7c60 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 msgr2=0x7f14040d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 0x7f14040d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 >> v1:172.21.15.138:6789/0 conn(0x7f14040d1160 legacy=0x7f14040b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14040ce100 con 0x7f14040b7c60 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f13fc0019f0 con 0x7f14040b7c60 2024-04-25T16:27:51.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 >> v1:172.21.15.94:6789/0 conn(0x7f14040b7c60 legacy=0x7f14040b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec003b70 0x7f13ec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ec004800 0x7f13ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 0x7f13ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f140415da40 con 0x7f13ec004800 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00bad0 con 0x7f13ec003b70 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec00bcd0 con 0x7f140415d630 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1402ffd640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ec004800 0x7f13ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f14037fe640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec003b70 0x7f13ec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1403fff640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 0x7f13ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1402ffd640 1 -- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 msgr2=0x7f13ec005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:51.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1402ffd640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140415d630 0x7f13ec005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1402ffd640 1 -- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec003b70 msgr2=0x7f13ec003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1402ffd640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec003b70 0x7f13ec003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.428+0000 7f1402ffd640 1 -- 172.21.15.94:0/1520894107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec003520 con 0x7f13ec004800 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.429+0000 7f1402ffd640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ec004800 0x7f13ec004c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f13f80018d0 tx=0x7f13ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f7baadb6237534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.429+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80014f0 con 0x7f13ec004800 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.429+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f80052d0 con 0x7f13ec004800 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.429+0000 7f14027fc640 1 -- 172.21.15.94:0/1520894107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f802aca0 con 0x7f13ec004800 2024-04-25T16:27:51.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1409d38640 1 -- 172.21.15.94:0/1520894107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ec004800 msgr2=0x7f13ec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1409d38640 1 --2- 172.21.15.94:0/1520894107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ec004800 0x7f13ec004c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f13f80018d0 tx=0x7f13ec005830 comp rx=0 tx=0).stop 2024-04-25T16:27:51.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1403fff640 1 -- 172.21.15.94:0/1520894107 reap_dead start 2024-04-25T16:27:51.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1409d38640 1 -- 172.21.15.94:0/1520894107 shutdown_connections 2024-04-25T16:27:51.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1409d38640 1 -- 172.21.15.94:0/1520894107 >> 172.21.15.94:0/1520894107 conn(0x7f140409b9b0 msgr2=0x7f140409bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:51.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1409d38640 1 -- 172.21.15.94:0/1520894107 shutdown_connections 2024-04-25T16:27:51.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.430+0000 7f1409d38640 1 -- 172.21.15.94:0/1520894107 wait complete. 2024-04-25T16:27:51.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.432+0000 7f1409d38640 1 Processor -- start 2024-04-25T16:27:51.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.432+0000 7f1409d38640 1 -- start start 2024-04-25T16:27:51.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1409d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec004800 0x7f140415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1409d38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1404157670 0x7f140415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1409d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 0x7f14040cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1409d38640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14040d05c0 con 0x7f14040cb990 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1409d38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14040d0440 con 0x7f13ec004800 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1409d38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f14040d0c20 con 0x7f1404157670 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 0x7f14040cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 0x7f14040cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42496/0 (socket says 172.21.15.94:42496) 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 -- 172.21.15.94:0/404176005 learned_addr learned my addr 172.21.15.94:0/404176005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1402ffd640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1404157670 0x7f140415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f14037fe640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec004800 0x7f140415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 -- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1404157670 msgr2=0x7f140415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1404157670 0x7f140415c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 -- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec004800 msgr2=0x7f140415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.433+0000 7f1403fff640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec004800 0x7f140415ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f1403fff640 1 -- 172.21.15.94:0/404176005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14040cc2e0 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f1403fff640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 0x7f14040cbda0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f13ec004780 tx=0x7f13fc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00ab90 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc019070 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1404160a00 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0133f0 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.434+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140406e2c0 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.435+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140406a180 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.436+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f13fc013590 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.436+0000 7f13ebfff640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f13f00744b0 0x7f13f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.436+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f13fc095b80 con 0x7f14040cb990 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.436+0000 7f14037fe640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f13f00744b0 0x7f13f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:51.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.437+0000 7f14037fe640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f13f00744b0 0x7f13f00769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009160 tx=0x7f13f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:51.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.441+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14040cc2e0 con 0x7f14040cb990 2024-04-25T16:27:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.630+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14040b2ca0 con 0x7f14040cb990 2024-04-25T16:27:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.631+0000 7f13ebfff640 1 -- 172.21.15.94:0/404176005 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f13fc05ffa0 con 0x7f14040cb990 2024-04-25T16:27:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:27:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.632+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f13f00744b0 msgr2=0x7f13f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f13f00744b0 0x7f13f00769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009160 tx=0x7f13f4008040 comp rx=0 tx=0).stop 2024-04-25T16:27:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 msgr2=0x7f14040cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 0x7f14040cbda0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f13ec004780 tx=0x7f13fc007bd0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 shutdown_connections 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f13f00744b0 0x7f13f00769c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14040cb990 0x7f14040cbda0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1404157670 0x7f140415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 --2- 172.21.15.94:0/404176005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13ec004800 0x7f140415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 >> 172.21.15.94:0/404176005 conn(0x7f140409b9b0 msgr2=0x7f140409bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 shutdown_connections 2024-04-25T16:27:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:51.633+0000 7f1409d38640 1 -- 172.21.15.94:0/404176005 wait complete. 2024-04-25T16:27:51.644 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T16:27:51.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.3 2024-04-25T16:27:51.677 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:27:51.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.677 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T16:27:51.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.735 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:51.750 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T16:27:51.750 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T16:27:51.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.750 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:27:51.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.814 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:27:51.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T16:27:51.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.979 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:27:51.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:51.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T16:27:52.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:27:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-04-25T16:27:52.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:52.073 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:27:52.105 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T16:27:52.105 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T16:27:52.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:52.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:27:52.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:52.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:27:52.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:52.258 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T16:27:52.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:27:52.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:27:52.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:52.368 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:27:52.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:27:52.425 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T16:27:52.524 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-04-25T16:27:52.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:27:52.525 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T16:27:52.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T16:27:52.567 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.675+0000 7f05e38e0640 1 Processor -- start 2024-04-25T16:27:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.675+0000 7f05e38e0640 1 -- start start 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e38e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 0x7f05dc09bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e38e0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05dc06d2a0 0x7f05dc06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e38e0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05dc0b7fa0 con 0x7f05dc0b37a0 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e38e0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f05dc06c570 con 0x7f05dc06d2a0 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e38e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc06c7c0 con 0x7f05dc06e590 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e0e7b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f05dc0b37a0 0x7f05dc0b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48004/0 (socket says 172.21.15.94:48004) 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e0e7b640 1 -- 172.21.15.94:0/1875274610 learned_addr learned my addr 172.21.15.94:0/1875274610 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e1e7d640 1 --2- 172.21.15.94:0/1875274610 >> v2:172.21.15.138:3300/0 conn(0x7f05dc06d2a0 0x7f05dc06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05e167c640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 0x7f05dc09bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3125063439 0 0) 0x7f05dc0b7fa0 con 0x7f05dc0b37a0 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05c8003520 con 0x7f05dc0b37a0 2024-04-25T16:27:52.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3816706647 0 0) 0x7f05c8003520 con 0x7f05dc0b37a0 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 msgr2=0x7f05dc09bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 0x7f05dc09bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 >> v2:172.21.15.138:3300/0 conn(0x7f05dc06d2a0 msgr2=0x7f05dc06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 --2- 172.21.15.94:0/1875274610 >> v2:172.21.15.138:3300/0 conn(0x7f05dc06d2a0 0x7f05dc06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc06caa0 con 0x7f05dc0b37a0 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f05cc001670 con 0x7f05dc0b37a0 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.677+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 >> v1:172.21.15.94:6789/0 conn(0x7f05dc0b37a0 legacy=0x7f05dc0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05d3fff640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c80039b0 0x7f05c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05d3fff640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 0x7f05c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05d3fff640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8004d40 0x7f05c8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc06c570 con 0x7f05c80039b0 2024-04-25T16:27:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc06c7c0 con 0x7f05c8004d40 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05c800ba40 con 0x7f05dc06e590 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e167c640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c80039b0 0x7f05c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e0e7b640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 0x7f05c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e1e7d640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8004d40 0x7f05c8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e167c640 1 -- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 msgr2=0x7f05c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e167c640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc06e590 0x7f05c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e167c640 1 -- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8004d40 msgr2=0x7f05c8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e167c640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8004d40 0x7f05c8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.678+0000 7f05e167c640 1 -- 172.21.15.94:0/1875274610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c8003520 con 0x7f05c80039b0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05e167c640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c80039b0 0x7f05c8003e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0cd9f0 tx=0x7f05c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ce7ae2a094c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8009c10 con 0x7f05c80039b0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d8036070 con 0x7f05c80039b0 2024-04-25T16:27:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05d3fff640 1 -- 172.21.15.94:0/1875274610 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d80317f0 con 0x7f05c80039b0 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05e38e0640 1 -- 172.21.15.94:0/1875274610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c80039b0 msgr2=0x7f05c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1875274610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c80039b0 0x7f05c8003e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0cd9f0 tx=0x7f05c8005830 comp rx=0 tx=0).stop 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05e1e7d640 1 -- 172.21.15.94:0/1875274610 reap_dead start 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05e38e0640 1 -- 172.21.15.94:0/1875274610 shutdown_connections 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.679+0000 7f05e38e0640 1 -- 172.21.15.94:0/1875274610 >> 172.21.15.94:0/1875274610 conn(0x7f05dc09b9b0 msgr2=0x7f05dc0b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.680+0000 7f05e38e0640 1 -- 172.21.15.94:0/1875274610 shutdown_connections 2024-04-25T16:27:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.680+0000 7f05e38e0640 1 -- 172.21.15.94:0/1875274610 wait complete. 2024-04-25T16:27:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.681+0000 7f05e38e0640 1 Processor -- start 2024-04-25T16:27:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.681+0000 7f05e38e0640 1 -- start start 2024-04-25T16:27:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80039b0 0x7f05dc1fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 0x7f05dc0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc0cbef0 0x7f05dc15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc0af5e0 con 0x7f05dc14d0d0 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc0b7c50 con 0x7f05c80039b0 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc161d40 con 0x7f05dc0cbef0 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 0x7f05dc0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 0x7f05dc0cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42504/0 (socket says 172.21.15.94:42504) 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 -- 172.21.15.94:0/1248883581 learned_addr learned my addr 172.21.15.94:0/1248883581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 -- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc0cbef0 msgr2=0x7f05dc15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc0cbef0 0x7f05dc15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 -- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80039b0 msgr2=0x7f05dc1fafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e167c640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80039b0 0x7f05dc1fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80039b0 0x7f05dc1fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 -- 172.21.15.94:0/1248883581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc0cc300 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e0e7b640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 0x7f05dc0cb9b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f05cc007290 tx=0x7f05cc007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc01a070 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05cc00f9c0 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc0d14e0 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.683+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00fdb0 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.684+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc0d1aa0 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.685+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc06a180 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.685+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f05cc01d070 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.686+0000 7f05d1ffb640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f05c00744e0 0x7f05c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.686+0000 7f05e167c640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f05c00744e0 0x7f05c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.686+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f05cc095180 con 0x7f05dc14d0d0 2024-04-25T16:27:52.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.686+0000 7f05e167c640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f05c00744e0 0x7f05c00769f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05d8035000 tx=0x7f05d802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:52.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.690+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05dc0cc300 con 0x7f05dc14d0d0 2024-04-25T16:27:52.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.863+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05dc0b40e0 con 0x7f05c00744e0 2024-04-25T16:27:52.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.871+0000 7f05d1ffb640 1 -- 172.21.15.94:0/1248883581 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f05dc0b40e0 con 0x7f05c00744e0 2024-04-25T16:27:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:52.877 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f05c00744e0 msgr2=0x7f05c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f05c00744e0 0x7f05c00769f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05d8035000 tx=0x7f05d802f000 comp rx=0 tx=0).stop 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 msgr2=0x7f05dc0cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 0x7f05dc0cb9b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f05cc007290 tx=0x7f05cc007690 comp rx=0 tx=0).stop 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 shutdown_connections 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f05c00744e0 0x7f05c00769f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05dc0cbef0 0x7f05dc15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05dc14d0d0 0x7f05dc0cb9b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 --2- 172.21.15.94:0/1248883581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80039b0 0x7f05dc1fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:52.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 >> 172.21.15.94:0/1248883581 conn(0x7f05dc09b9b0 msgr2=0x7f05dc160700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:52.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.874+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 shutdown_connections 2024-04-25T16:27:52.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:52.875+0000 7f05e38e0640 1 -- 172.21.15.94:0/1248883581 wait complete. 2024-04-25T16:27:52.886 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-04-25T16:27:51.309947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605699,"num_objects":25,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":166,"num_write_kb":1665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236276,"kb_used_data":3484,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738444,"statfs":{"total":773094113280,"available":772852166656,"internally_reserved":0,"allocated":3567616,"data_stored":2152328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10179,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699051+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:36.699051+0000","last_peered":"2024-04-25T16:27:36.699051+0000","last_clean":"2024-04-25T16:27:36.699051+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:36.699051+0000","last_undegraded":"2024-04-25T16:27:36.699051+0000","last_fullsized":"2024-04-25T16:27:36.699051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694756+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:36.694756+0000","last_peered":"2024-04-25T16:27:36.694756+0000","last_clean":"2024-04-25T16:27:36.694756+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:36.694756+0000","last_undegraded":"2024-04-25T16:27:36.694756+0000","last_fullsized":"2024-04-25T16:27:36.694756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693795+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:36.693795+0000","last_peered":"2024-04-25T16:27:36.693795+0000","last_clean":"2024-04-25T16:27:36.693795+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:36.693795+0000","last_undegraded":"2024-04-25T16:27:36.693795+0000","last_fullsized":"2024-04-25T16:27:36.693795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694849+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:36.694849+0000","last_peered":"2024-04-25T16:27:36.694849+0000","last_clean":"2024-04-25T16:27:36.694849+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:36.694849+0000","last_undegraded":"2024-04-25T16:27:36.694849+0000","last_fullsized":"2024-04-25T16:27:36.694849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697915+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:36.697915+0000","last_peered":"2024-04-25T16:27:36.697915+0000","last_clean":"2024-04-25T16:27:36.697915+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:36.697915+0000","last_undegraded":"2024-04-25T16:27:36.697915+0000","last_fullsized":"2024-04-25T16:27:36.697915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750192+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:36.750192+0000","last_peered":"2024-04-25T16:27:36.750192+0000","last_clean":"2024-04-25T16:27:36.750192+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:36.750192+0000","last_undegraded":"2024-04-25T16:27:36.750192+0000","last_fullsized":"2024-04-25T16:27:36.750192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699035+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:36.699035+0000","last_peered":"2024-04-25T16:27:36.699035+0000","last_clean":"2024-04-25T16:27:36.699035+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:36.699035+0000","last_undegraded":"2024-04-25T16:27:36.699035+0000","last_fullsized":"2024-04-25T16:27:36.699035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764530+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:36.764530+0000","last_peered":"2024-04-25T16:27:36.764530+0000","last_clean":"2024-04-25T16:27:36.764530+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:36.764530+0000","last_undegraded":"2024-04-25T16:27:36.764530+0000","last_fullsized":"2024-04-25T16:27:36.764530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750366+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:36.750366+0000","last_peered":"2024-04-25T16:27:36.750366+0000","last_clean":"2024-04-25T16:27:36.750366+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:36.750366+0000","last_undegraded":"2024-04-25T16:27:36.750366+0000","last_fullsized":"2024-04-25T16:27:36.750366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697999+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:36.697999+0000","last_peered":"2024-04-25T16:27:36.697999+0000","last_clean":"2024-04-25T16:27:36.697999+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:36.697999+0000","last_undegraded":"2024-04-25T16:27:36.697999+0000","last_fullsized":"2024-04-25T16:27:36.697999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764669+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:36.764669+0000","last_peered":"2024-04-25T16:27:36.764669+0000","last_clean":"2024-04-25T16:27:36.764669+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:36.764669+0000","last_undegraded":"2024-04-25T16:27:36.764669+0000","last_fullsized":"2024-04-25T16:27:36.764669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750279+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:36.750279+0000","last_peered":"2024-04-25T16:27:36.750279+0000","last_clean":"2024-04-25T16:27:36.750279+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:36.750279+0000","last_undegraded":"2024-04-25T16:27:36.750279+0000","last_fullsized":"2024-04-25T16:27:36.750279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697286+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:36.697286+0000","last_peered":"2024-04-25T16:27:36.697286+0000","last_clean":"2024-04-25T16:27:36.697286+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:36.697286+0000","last_undegraded":"2024-04-25T16:27:36.697286+0000","last_fullsized":"2024-04-25T16:27:36.697286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698555+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:36.698555+0000","last_peered":"2024-04-25T16:27:36.698555+0000","last_clean":"2024-04-25T16:27:36.698555+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:36.698555+0000","last_undegraded":"2024-04-25T16:27:36.698555+0000","last_fullsized":"2024-04-25T16:27:36.698555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750951+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:36.750951+0000","last_peered":"2024-04-25T16:27:36.750951+0000","last_clean":"2024-04-25T16:27:36.750951+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:36.750951+0000","last_undegraded":"2024-04-25T16:27:36.750951+0000","last_fullsized":"2024-04-25T16:27:36.750951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694265+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:36.694265+0000","last_peered":"2024-04-25T16:27:36.694265+0000","last_clean":"2024-04-25T16:27:36.694265+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:36.694265+0000","last_undegraded":"2024-04-25T16:27:36.694265+0000","last_fullsized":"2024-04-25T16:27:36.694265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697095+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:36.697095+0000","last_peered":"2024-04-25T16:27:36.697095+0000","last_clean":"2024-04-25T16:27:36.697095+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:36.697095+0000","last_undegraded":"2024-04-25T16:27:36.697095+0000","last_fullsized":"2024-04-25T16:27:36.697095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750759+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:36.750759+0000","last_peered":"2024-04-25T16:27:36.750759+0000","last_clean":"2024-04-25T16:27:36.750759+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:36.750759+0000","last_undegraded":"2024-04-25T16:27:36.750759+0000","last_fullsized":"2024-04-25T16:27:36.750759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694142+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:36.694142+0000","last_peered":"2024-04-25T16:27:36.694142+0000","last_clean":"2024-04-25T16:27:36.694142+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:36.694142+0000","last_undegraded":"2024-04-25T16:27:36.694142+0000","last_fullsized":"2024-04-25T16:27:36.694142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696965+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:36.696965+0000","last_peered":"2024-04-25T16:27:36.696965+0000","last_clean":"2024-04-25T16:27:36.696965+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:36.696965+0000","last_undegraded":"2024-04-25T16:27:36.696965+0000","last_fullsized":"2024-04-25T16:27:36.696965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.752445+0000","last_change":"2024-04-25T16:27:36.751906+0000","last_active":"2024-04-25T16:27:36.752445+0000","last_peered":"2024-04-25T16:27:36.752445+0000","last_clean":"2024-04-25T16:27:36.752445+0000","last_became_active":"2024-04-25T16:27:36.751463+0000","last_became_peered":"2024-04-25T16:27:36.751463+0000","last_unstale":"2024-04-25T16:27:36.752445+0000","last_undegraded":"2024-04-25T16:27:36.752445+0000","last_fullsized":"2024-04-25T16:27:36.752445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750568+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:36.750568+0000","last_peered":"2024-04-25T16:27:36.750568+0000","last_clean":"2024-04-25T16:27:36.750568+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:36.750568+0000","last_undegraded":"2024-04-25T16:27:36.750568+0000","last_fullsized":"2024-04-25T16:27:36.750568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697515+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:36.697515+0000","last_peered":"2024-04-25T16:27:36.697515+0000","last_clean":"2024-04-25T16:27:36.697515+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:36.697515+0000","last_undegraded":"2024-04-25T16:27:36.697515+0000","last_fullsized":"2024-04-25T16:27:36.697515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750847+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:36.750847+0000","last_peered":"2024-04-25T16:27:36.750847+0000","last_clean":"2024-04-25T16:27:36.750847+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:36.750847+0000","last_undegraded":"2024-04-25T16:27:36.750847+0000","last_fullsized":"2024-04-25T16:27:36.750847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698515+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:36.698515+0000","last_peered":"2024-04-25T16:27:36.698515+0000","last_clean":"2024-04-25T16:27:36.698515+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:36.698515+0000","last_undegraded":"2024-04-25T16:27:36.698515+0000","last_fullsized":"2024-04-25T16:27:36.698515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.700039+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:36.700039+0000","last_peered":"2024-04-25T16:27:36.700039+0000","last_clean":"2024-04-25T16:27:36.700039+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:36.700039+0000","last_undegraded":"2024-04-25T16:27:36.700039+0000","last_fullsized":"2024-04-25T16:27:36.700039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750332+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:36.750332+0000","last_peered":"2024-04-25T16:27:36.750332+0000","last_clean":"2024-04-25T16:27:36.750332+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:36.750332+0000","last_undegraded":"2024-04-25T16:27:36.750332+0000","last_fullsized":"2024-04-25T16:27:36.750332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.695158+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:36.695158+0000","last_peered":"2024-04-25T16:27:36.695158+0000","last_clean":"2024-04-25T16:27:36.695158+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:36.695158+0000","last_undegraded":"2024-04-25T16:27:36.695158+0000","last_fullsized":"2024-04-25T16:27:36.695158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:41.552216+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:41.552216+0000","last_peered":"2024-04-25T16:27:41.552216+0000","last_clean":"2024-04-25T16:27:41.552216+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:41.552216+0000","last_undegraded":"2024-04-25T16:27:41.552216+0000","last_fullsized":"2024-04-25T16:27:41.552216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693728+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:36.693728+0000","last_peered":"2024-04-25T16:27:36.693728+0000","last_clean":"2024-04-25T16:27:36.693728+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:36.693728+0000","last_undegraded":"2024-04-25T16:27:36.693728+0000","last_fullsized":"2024-04-25T16:27:36.693728+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697033+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:36.697033+0000","last_peered":"2024-04-25T16:27:36.697033+0000","last_clean":"2024-04-25T16:27:36.697033+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:36.697033+0000","last_undegraded":"2024-04-25T16:27:36.697033+0000","last_fullsized":"2024-04-25T16:27:36.697033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698561+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:36.698561+0000","last_peered":"2024-04-25T16:27:36.698561+0000","last_clean":"2024-04-25T16:27:36.698561+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:36.698561+0000","last_undegraded":"2024-04-25T16:27:36.698561+0000","last_fullsized":"2024-04-25T16:27:36.698561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698300+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:36.698300+0000","last_peered":"2024-04-25T16:27:36.698300+0000","last_clean":"2024-04-25T16:27:36.698300+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:36.698300+0000","last_undegraded":"2024-04-25T16:27:36.698300+0000","last_fullsized":"2024-04-25T16:27:36.698300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764769+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:36.764769+0000","last_peered":"2024-04-25T16:27:36.764769+0000","last_clean":"2024-04-25T16:27:36.764769+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:36.764769+0000","last_undegraded":"2024-04-25T16:27:36.764769+0000","last_fullsized":"2024-04-25T16:27:36.764769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698328+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:36.698328+0000","last_peered":"2024-04-25T16:27:36.698328+0000","last_clean":"2024-04-25T16:27:36.698328+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:36.698328+0000","last_undegraded":"2024-04-25T16:27:36.698328+0000","last_fullsized":"2024-04-25T16:27:36.698328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696738+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:36.696738+0000","last_peered":"2024-04-25T16:27:36.696738+0000","last_clean":"2024-04-25T16:27:36.696738+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:36.696738+0000","last_undegraded":"2024-04-25T16:27:36.696738+0000","last_fullsized":"2024-04-25T16:27:36.696738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750724+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:36.750724+0000","last_peered":"2024-04-25T16:27:36.750724+0000","last_clean":"2024-04-25T16:27:36.750724+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:36.750724+0000","last_undegraded":"2024-04-25T16:27:36.750724+0000","last_fullsized":"2024-04-25T16:27:36.750724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764844+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:36.764844+0000","last_peered":"2024-04-25T16:27:36.764844+0000","last_clean":"2024-04-25T16:27:36.764844+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:36.764844+0000","last_undegraded":"2024-04-25T16:27:36.764844+0000","last_fullsized":"2024-04-25T16:27:36.764844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696930+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:36.696930+0000","last_peered":"2024-04-25T16:27:36.696930+0000","last_clean":"2024-04-25T16:27:36.696930+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:36.696930+0000","last_undegraded":"2024-04-25T16:27:36.696930+0000","last_fullsized":"2024-04-25T16:27:36.696930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698136+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:36.698136+0000","last_peered":"2024-04-25T16:27:36.698136+0000","last_clean":"2024-04-25T16:27:36.698136+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:36.698136+0000","last_undegraded":"2024-04-25T16:27:36.698136+0000","last_fullsized":"2024-04-25T16:27:36.698136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697778+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:36.697778+0000","last_peered":"2024-04-25T16:27:36.697778+0000","last_clean":"2024-04-25T16:27:36.697778+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:36.697778+0000","last_undegraded":"2024-04-25T16:27:36.697778+0000","last_fullsized":"2024-04-25T16:27:36.697778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697213+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:36.697213+0000","last_peered":"2024-04-25T16:27:36.697213+0000","last_clean":"2024-04-25T16:27:36.697213+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:36.697213+0000","last_undegraded":"2024-04-25T16:27:36.697213+0000","last_fullsized":"2024-04-25T16:27:36.697213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697266+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:36.697266+0000","last_peered":"2024-04-25T16:27:36.697266+0000","last_clean":"2024-04-25T16:27:36.697266+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:36.697266+0000","last_undegraded":"2024-04-25T16:27:36.697266+0000","last_fullsized":"2024-04-25T16:27:36.697266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697357+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:36.697357+0000","last_peered":"2024-04-25T16:27:36.697357+0000","last_clean":"2024-04-25T16:27:36.697357+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:36.697357+0000","last_undegraded":"2024-04-25T16:27:36.697357+0000","last_fullsized":"2024-04-25T16:27:36.697357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698254+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:36.698254+0000","last_peered":"2024-04-25T16:27:36.698254+0000","last_clean":"2024-04-25T16:27:36.698254+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:36.698254+0000","last_undegraded":"2024-04-25T16:27:36.698254+0000","last_fullsized":"2024-04-25T16:27:36.698254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694153+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:36.694153+0000","last_peered":"2024-04-25T16:27:36.694153+0000","last_clean":"2024-04-25T16:27:36.694153+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:36.694153+0000","last_undegraded":"2024-04-25T16:27:36.694153+0000","last_fullsized":"2024-04-25T16:27:36.694153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764525+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:36.764525+0000","last_peered":"2024-04-25T16:27:36.764525+0000","last_clean":"2024-04-25T16:27:36.764525+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:36.764525+0000","last_undegraded":"2024-04-25T16:27:36.764525+0000","last_fullsized":"2024-04-25T16:27:36.764525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697495+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:36.697495+0000","last_peered":"2024-04-25T16:27:36.697495+0000","last_clean":"2024-04-25T16:27:36.697495+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:36.697495+0000","last_undegraded":"2024-04-25T16:27:36.697495+0000","last_fullsized":"2024-04-25T16:27:36.697495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698810+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:36.698810+0000","last_peered":"2024-04-25T16:27:36.698810+0000","last_clean":"2024-04-25T16:27:36.698810+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:36.698810+0000","last_undegraded":"2024-04-25T16:27:36.698810+0000","last_fullsized":"2024-04-25T16:27:36.698810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698193+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:36.698193+0000","last_peered":"2024-04-25T16:27:36.698193+0000","last_clean":"2024-04-25T16:27:36.698193+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:36.698193+0000","last_undegraded":"2024-04-25T16:27:36.698193+0000","last_fullsized":"2024-04-25T16:27:36.698193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697778+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:36.697778+0000","last_peered":"2024-04-25T16:27:36.697778+0000","last_clean":"2024-04-25T16:27:36.697778+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:36.697778+0000","last_undegraded":"2024-04-25T16:27:36.697778+0000","last_fullsized":"2024-04-25T16:27:36.697778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697238+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:36.697238+0000","last_peered":"2024-04-25T16:27:36.697238+0000","last_clean":"2024-04-25T16:27:36.697238+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:36.697238+0000","last_undegraded":"2024-04-25T16:27:36.697238+0000","last_fullsized":"2024-04-25T16:27:36.697238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764579+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:36.764579+0000","last_peered":"2024-04-25T16:27:36.764579+0000","last_clean":"2024-04-25T16:27:36.764579+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:36.764579+0000","last_undegraded":"2024-04-25T16:27:36.764579+0000","last_fullsized":"2024-04-25T16:27:36.764579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697603+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:36.697603+0000","last_peered":"2024-04-25T16:27:36.697603+0000","last_clean":"2024-04-25T16:27:36.697603+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:36.697603+0000","last_undegraded":"2024-04-25T16:27:36.697603+0000","last_fullsized":"2024-04-25T16:27:36.697603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698509+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:36.698509+0000","last_peered":"2024-04-25T16:27:36.698509+0000","last_clean":"2024-04-25T16:27:36.698509+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:36.698509+0000","last_undegraded":"2024-04-25T16:27:36.698509+0000","last_fullsized":"2024-04-25T16:27:36.698509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699318+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:36.699318+0000","last_peered":"2024-04-25T16:27:36.699318+0000","last_clean":"2024-04-25T16:27:36.699318+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:36.699318+0000","last_undegraded":"2024-04-25T16:27:36.699318+0000","last_fullsized":"2024-04-25T16:27:36.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697295+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:36.697295+0000","last_peered":"2024-04-25T16:27:36.697295+0000","last_clean":"2024-04-25T16:27:36.697295+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:36.697295+0000","last_undegraded":"2024-04-25T16:27:36.697295+0000","last_fullsized":"2024-04-25T16:27:36.697295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.763861+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:36.763861+0000","last_peered":"2024-04-25T16:27:36.763861+0000","last_clean":"2024-04-25T16:27:36.763861+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:36.763861+0000","last_undegraded":"2024-04-25T16:27:36.763861+0000","last_fullsized":"2024-04-25T16:27:36.763861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698846+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:36.698846+0000","last_peered":"2024-04-25T16:27:36.698846+0000","last_clean":"2024-04-25T16:27:36.698846+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:36.698846+0000","last_undegraded":"2024-04-25T16:27:36.698846+0000","last_fullsized":"2024-04-25T16:27:36.698846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751170+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:36.751170+0000","last_peered":"2024-04-25T16:27:36.751170+0000","last_clean":"2024-04-25T16:27:36.751170+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:36.751170+0000","last_undegraded":"2024-04-25T16:27:36.751170+0000","last_fullsized":"2024-04-25T16:27:36.751170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750149+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:36.750149+0000","last_peered":"2024-04-25T16:27:36.750149+0000","last_clean":"2024-04-25T16:27:36.750149+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:36.750149+0000","last_undegraded":"2024-04-25T16:27:36.750149+0000","last_fullsized":"2024-04-25T16:27:36.750149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698175+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:36.698175+0000","last_peered":"2024-04-25T16:27:36.698175+0000","last_clean":"2024-04-25T16:27:36.698175+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:36.698175+0000","last_undegraded":"2024-04-25T16:27:36.698175+0000","last_fullsized":"2024-04-25T16:27:36.698175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697542+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:36.697542+0000","last_peered":"2024-04-25T16:27:36.697542+0000","last_clean":"2024-04-25T16:27:36.697542+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:36.697542+0000","last_undegraded":"2024-04-25T16:27:36.697542+0000","last_fullsized":"2024-04-25T16:27:36.697542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698129+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:36.698129+0000","last_peered":"2024-04-25T16:27:36.698129+0000","last_clean":"2024-04-25T16:27:36.698129+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:36.698129+0000","last_undegraded":"2024-04-25T16:27:36.698129+0000","last_fullsized":"2024-04-25T16:27:36.698129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697530+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:36.697530+0000","last_peered":"2024-04-25T16:27:36.697530+0000","last_clean":"2024-04-25T16:27:36.697530+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:36.697530+0000","last_undegraded":"2024-04-25T16:27:36.697530+0000","last_fullsized":"2024-04-25T16:27:36.697530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697833+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:36.697833+0000","last_peered":"2024-04-25T16:27:36.697833+0000","last_clean":"2024-04-25T16:27:36.697833+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:36.697833+0000","last_undegraded":"2024-04-25T16:27:36.697833+0000","last_fullsized":"2024-04-25T16:27:36.697833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694937+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:36.694937+0000","last_peered":"2024-04-25T16:27:36.694937+0000","last_clean":"2024-04-25T16:27:36.694937+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:36.694937+0000","last_undegraded":"2024-04-25T16:27:36.694937+0000","last_fullsized":"2024-04-25T16:27:36.694937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698423+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:36.698423+0000","last_peered":"2024-04-25T16:27:36.698423+0000","last_clean":"2024-04-25T16:27:36.698423+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:36.698423+0000","last_undegraded":"2024-04-25T16:27:36.698423+0000","last_fullsized":"2024-04-25T16:27:36.698423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764677+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:36.764677+0000","last_peered":"2024-04-25T16:27:36.764677+0000","last_clean":"2024-04-25T16:27:36.764677+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:36.764677+0000","last_undegraded":"2024-04-25T16:27:36.764677+0000","last_fullsized":"2024-04-25T16:27:36.764677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694274+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:36.694274+0000","last_peered":"2024-04-25T16:27:36.694274+0000","last_clean":"2024-04-25T16:27:36.694274+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:36.694274+0000","last_undegraded":"2024-04-25T16:27:36.694274+0000","last_fullsized":"2024-04-25T16:27:36.694274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698335+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:36.698335+0000","last_peered":"2024-04-25T16:27:36.698335+0000","last_clean":"2024-04-25T16:27:36.698335+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:36.698335+0000","last_undegraded":"2024-04-25T16:27:36.698335+0000","last_fullsized":"2024-04-25T16:27:36.698335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698767+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:36.698767+0000","last_peered":"2024-04-25T16:27:36.698767+0000","last_clean":"2024-04-25T16:27:36.698767+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:36.698767+0000","last_undegraded":"2024-04-25T16:27:36.698767+0000","last_fullsized":"2024-04-25T16:27:36.698767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764506+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:36.764506+0000","last_peered":"2024-04-25T16:27:36.764506+0000","last_clean":"2024-04-25T16:27:36.764506+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:36.764506+0000","last_undegraded":"2024-04-25T16:27:36.764506+0000","last_fullsized":"2024-04-25T16:27:36.764506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698073+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:36.698073+0000","last_peered":"2024-04-25T16:27:36.698073+0000","last_clean":"2024-04-25T16:27:36.698073+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:36.698073+0000","last_undegraded":"2024-04-25T16:27:36.698073+0000","last_fullsized":"2024-04-25T16:27:36.698073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'21","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:47.513651+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:47.513651+0000","last_peered":"2024-04-25T16:27:47.513651+0000","last_clean":"2024-04-25T16:27:47.513651+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:47.513651+0000","last_undegraded":"2024-04-25T16:27:47.513651+0000","last_fullsized":"2024-04-25T16:27:47.513651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696855+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:36.696855+0000","last_peered":"2024-04-25T16:27:36.696855+0000","last_clean":"2024-04-25T16:27:36.696855+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:36.696855+0000","last_undegraded":"2024-04-25T16:27:36.696855+0000","last_fullsized":"2024-04-25T16:27:36.696855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.701265+0000","last_change":"2024-04-25T16:27:36.700437+0000","last_active":"2024-04-25T16:27:36.701265+0000","last_peered":"2024-04-25T16:27:36.701265+0000","last_clean":"2024-04-25T16:27:36.701265+0000","last_became_active":"2024-04-25T16:27:36.700105+0000","last_became_peered":"2024-04-25T16:27:36.700105+0000","last_unstale":"2024-04-25T16:27:36.701265+0000","last_undegraded":"2024-04-25T16:27:36.701265+0000","last_fullsized":"2024-04-25T16:27:36.701265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697453+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:36.697453+0000","last_peered":"2024-04-25T16:27:36.697453+0000","last_clean":"2024-04-25T16:27:36.697453+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:36.697453+0000","last_undegraded":"2024-04-25T16:27:36.697453+0000","last_fullsized":"2024-04-25T16:27:36.697453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750587+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:36.750587+0000","last_peered":"2024-04-25T16:27:36.750587+0000","last_clean":"2024-04-25T16:27:36.750587+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:36.750587+0000","last_undegraded":"2024-04-25T16:27:36.750587+0000","last_fullsized":"2024-04-25T16:27:36.750587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698936+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:36.698936+0000","last_peered":"2024-04-25T16:27:36.698936+0000","last_clean":"2024-04-25T16:27:36.698936+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:36.698936+0000","last_undegraded":"2024-04-25T16:27:36.698936+0000","last_fullsized":"2024-04-25T16:27:36.698936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764563+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:36.764563+0000","last_peered":"2024-04-25T16:27:36.764563+0000","last_clean":"2024-04-25T16:27:36.764563+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:36.764563+0000","last_undegraded":"2024-04-25T16:27:36.764563+0000","last_fullsized":"2024-04-25T16:27:36.764563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:41.196000+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:41.196000+0000","last_peered":"2024-04-25T16:27:41.196000+0000","last_clean":"2024-04-25T16:27:41.196000+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:41.196000+0000","last_undegraded":"2024-04-25T16:27:41.196000+0000","last_fullsized":"2024-04-25T16:27:41.196000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764463+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:36.764463+0000","last_peered":"2024-04-25T16:27:36.764463+0000","last_clean":"2024-04-25T16:27:36.764463+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:36.764463+0000","last_undegraded":"2024-04-25T16:27:36.764463+0000","last_fullsized":"2024-04-25T16:27:36.764463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699948+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:36.699948+0000","last_peered":"2024-04-25T16:27:36.699948+0000","last_clean":"2024-04-25T16:27:36.699948+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:36.699948+0000","last_undegraded":"2024-04-25T16:27:36.699948+0000","last_fullsized":"2024-04-25T16:27:36.699948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693864+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:36.693864+0000","last_peered":"2024-04-25T16:27:36.693864+0000","last_clean":"2024-04-25T16:27:36.693864+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:36.693864+0000","last_undegraded":"2024-04-25T16:27:36.693864+0000","last_fullsized":"2024-04-25T16:27:36.693864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.702563+0000","last_change":"2024-04-25T16:27:36.702563+0000","last_active":"2024-04-25T16:27:36.702563+0000","last_peered":"2024-04-25T16:27:36.702563+0000","last_clean":"2024-04-25T16:27:36.702563+0000","last_became_active":"2024-04-25T16:27:36.702077+0000","last_became_peered":"2024-04-25T16:27:36.702077+0000","last_unstale":"2024-04-25T16:27:36.702563+0000","last_undegraded":"2024-04-25T16:27:36.702563+0000","last_fullsized":"2024-04-25T16:27:36.702563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:48.724249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,2],"acting":[0,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.701750+0000","last_change":"2024-04-25T16:27:36.701750+0000","last_active":"2024-04-25T16:27:36.701750+0000","last_peered":"2024-04-25T16:27:36.701750+0000","last_clean":"2024-04-25T16:27:36.701750+0000","last_became_active":"2024-04-25T16:27:36.701427+0000","last_became_peered":"2024-04-25T16:27:36.701427+0000","last_unstale":"2024-04-25T16:27:36.701750+0000","last_undegraded":"2024-04-25T16:27:36.701750+0000","last_fullsized":"2024-04-25T16:27:36.701750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:44.695661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764137+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:36.764137+0000","last_peered":"2024-04-25T16:27:36.764137+0000","last_clean":"2024-04-25T16:27:36.764137+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:36.764137+0000","last_undegraded":"2024-04-25T16:27:36.764137+0000","last_fullsized":"2024-04-25T16:27:36.764137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.766198+0000","last_change":"2024-04-25T16:27:36.766198+0000","last_active":"2024-04-25T16:27:36.766198+0000","last_peered":"2024-04-25T16:27:36.766198+0000","last_clean":"2024-04-25T16:27:36.766198+0000","last_became_active":"2024-04-25T16:27:36.765669+0000","last_became_peered":"2024-04-25T16:27:36.765669+0000","last_unstale":"2024-04-25T16:27:36.766198+0000","last_undegraded":"2024-04-25T16:27:36.766198+0000","last_fullsized":"2024-04-25T16:27:36.766198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:30.911634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694021+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:36.694021+0000","last_peered":"2024-04-25T16:27:36.694021+0000","last_clean":"2024-04-25T16:27:36.694021+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:36.694021+0000","last_undegraded":"2024-04-25T16:27:36.694021+0000","last_fullsized":"2024-04-25T16:27:36.694021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694344+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:36.694344+0000","last_peered":"2024-04-25T16:27:36.694344+0000","last_clean":"2024-04-25T16:27:36.694344+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:36.694344+0000","last_undegraded":"2024-04-25T16:27:36.694344+0000","last_fullsized":"2024-04-25T16:27:36.694344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698727+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:36.698727+0000","last_peered":"2024-04-25T16:27:36.698727+0000","last_clean":"2024-04-25T16:27:36.698727+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:36.698727+0000","last_undegraded":"2024-04-25T16:27:36.698727+0000","last_fullsized":"2024-04-25T16:27:36.698727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698703+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:36.698703+0000","last_peered":"2024-04-25T16:27:36.698703+0000","last_clean":"2024-04-25T16:27:36.698703+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:36.698703+0000","last_undegraded":"2024-04-25T16:27:36.698703+0000","last_fullsized":"2024-04-25T16:27:36.698703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697541+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:36.697541+0000","last_peered":"2024-04-25T16:27:36.697541+0000","last_clean":"2024-04-25T16:27:36.697541+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:36.697541+0000","last_undegraded":"2024-04-25T16:27:36.697541+0000","last_fullsized":"2024-04-25T16:27:36.697541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694183+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:36.694183+0000","last_peered":"2024-04-25T16:27:36.694183+0000","last_clean":"2024-04-25T16:27:36.694183+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:36.694183+0000","last_undegraded":"2024-04-25T16:27:36.694183+0000","last_fullsized":"2024-04-25T16:27:36.694183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750593+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:36.750593+0000","last_peered":"2024-04-25T16:27:36.750593+0000","last_clean":"2024-04-25T16:27:36.750593+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:36.750593+0000","last_undegraded":"2024-04-25T16:27:36.750593+0000","last_fullsized":"2024-04-25T16:27:36.750593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694424+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:36.694424+0000","last_peered":"2024-04-25T16:27:36.694424+0000","last_clean":"2024-04-25T16:27:36.694424+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:36.694424+0000","last_undegraded":"2024-04-25T16:27:36.694424+0000","last_fullsized":"2024-04-25T16:27:36.694424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697403+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:36.697403+0000","last_peered":"2024-04-25T16:27:36.697403+0000","last_clean":"2024-04-25T16:27:36.697403+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:36.697403+0000","last_undegraded":"2024-04-25T16:27:36.697403+0000","last_fullsized":"2024-04-25T16:27:36.697403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697750+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:36.697750+0000","last_peered":"2024-04-25T16:27:36.697750+0000","last_clean":"2024-04-25T16:27:36.697750+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:36.697750+0000","last_undegraded":"2024-04-25T16:27:36.697750+0000","last_fullsized":"2024-04-25T16:27:36.697750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.702267+0000","last_change":"2024-04-25T16:27:36.701488+0000","last_active":"2024-04-25T16:27:36.702267+0000","last_peered":"2024-04-25T16:27:36.702267+0000","last_clean":"2024-04-25T16:27:36.702267+0000","last_became_active":"2024-04-25T16:27:36.701156+0000","last_became_peered":"2024-04-25T16:27:36.701156+0000","last_unstale":"2024-04-25T16:27:36.702267+0000","last_undegraded":"2024-04-25T16:27:36.702267+0000","last_fullsized":"2024-04-25T16:27:36.702267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698662+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:36.698662+0000","last_peered":"2024-04-25T16:27:36.698662+0000","last_clean":"2024-04-25T16:27:36.698662+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:36.698662+0000","last_undegraded":"2024-04-25T16:27:36.698662+0000","last_fullsized":"2024-04-25T16:27:36.698662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694391+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:36.694391+0000","last_peered":"2024-04-25T16:27:36.694391+0000","last_clean":"2024-04-25T16:27:36.694391+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:36.694391+0000","last_undegraded":"2024-04-25T16:27:36.694391+0000","last_fullsized":"2024-04-25T16:27:36.694391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696935+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:36.696935+0000","last_peered":"2024-04-25T16:27:36.696935+0000","last_clean":"2024-04-25T16:27:36.696935+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:36.696935+0000","last_undegraded":"2024-04-25T16:27:36.696935+0000","last_fullsized":"2024-04-25T16:27:36.696935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699512+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:36.699512+0000","last_peered":"2024-04-25T16:27:36.699512+0000","last_clean":"2024-04-25T16:27:36.699512+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:36.699512+0000","last_undegraded":"2024-04-25T16:27:36.699512+0000","last_fullsized":"2024-04-25T16:27:36.699512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.695043+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:36.695043+0000","last_peered":"2024-04-25T16:27:36.695043+0000","last_clean":"2024-04-25T16:27:36.695043+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:36.695043+0000","last_undegraded":"2024-04-25T16:27:36.695043+0000","last_fullsized":"2024-04-25T16:27:36.695043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764019+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:36.764019+0000","last_peered":"2024-04-25T16:27:36.764019+0000","last_clean":"2024-04-25T16:27:36.764019+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:36.764019+0000","last_undegraded":"2024-04-25T16:27:36.764019+0000","last_fullsized":"2024-04-25T16:27:36.764019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694199+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:36.694199+0000","last_peered":"2024-04-25T16:27:36.694199+0000","last_clean":"2024-04-25T16:27:36.694199+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:36.694199+0000","last_undegraded":"2024-04-25T16:27:36.694199+0000","last_fullsized":"2024-04-25T16:27:36.694199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764266+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:36.764266+0000","last_peered":"2024-04-25T16:27:36.764266+0000","last_clean":"2024-04-25T16:27:36.764266+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:36.764266+0000","last_undegraded":"2024-04-25T16:27:36.764266+0000","last_fullsized":"2024-04-25T16:27:36.764266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698420+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:36.698420+0000","last_peered":"2024-04-25T16:27:36.698420+0000","last_clean":"2024-04-25T16:27:36.698420+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:36.698420+0000","last_undegraded":"2024-04-25T16:27:36.698420+0000","last_fullsized":"2024-04-25T16:27:36.698420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698766+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:36.698766+0000","last_peered":"2024-04-25T16:27:36.698766+0000","last_clean":"2024-04-25T16:27:36.698766+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:36.698766+0000","last_undegraded":"2024-04-25T16:27:36.698766+0000","last_fullsized":"2024-04-25T16:27:36.698766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698354+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:36.698354+0000","last_peered":"2024-04-25T16:27:36.698354+0000","last_clean":"2024-04-25T16:27:36.698354+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:36.698354+0000","last_undegraded":"2024-04-25T16:27:36.698354+0000","last_fullsized":"2024-04-25T16:27:36.698354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697177+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:36.697177+0000","last_peered":"2024-04-25T16:27:36.697177+0000","last_clean":"2024-04-25T16:27:36.697177+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:36.697177+0000","last_undegraded":"2024-04-25T16:27:36.697177+0000","last_fullsized":"2024-04-25T16:27:36.697177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764875+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:36.764875+0000","last_peered":"2024-04-25T16:27:36.764875+0000","last_clean":"2024-04-25T16:27:36.764875+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:36.764875+0000","last_undegraded":"2024-04-25T16:27:36.764875+0000","last_fullsized":"2024-04-25T16:27:36.764875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764030+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:36.764030+0000","last_peered":"2024-04-25T16:27:36.764030+0000","last_clean":"2024-04-25T16:27:36.764030+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:36.764030+0000","last_undegraded":"2024-04-25T16:27:36.764030+0000","last_fullsized":"2024-04-25T16:27:36.764030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696865+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:36.696865+0000","last_peered":"2024-04-25T16:27:36.696865+0000","last_clean":"2024-04-25T16:27:36.696865+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:36.696865+0000","last_undegraded":"2024-04-25T16:27:36.696865+0000","last_fullsized":"2024-04-25T16:27:36.696865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698226+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:36.698226+0000","last_peered":"2024-04-25T16:27:36.698226+0000","last_clean":"2024-04-25T16:27:36.698226+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:36.698226+0000","last_undegraded":"2024-04-25T16:27:36.698226+0000","last_fullsized":"2024-04-25T16:27:36.698226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694041+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:36.694041+0000","last_peered":"2024-04-25T16:27:36.694041+0000","last_clean":"2024-04-25T16:27:36.694041+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:36.694041+0000","last_undegraded":"2024-04-25T16:27:36.694041+0000","last_fullsized":"2024-04-25T16:27:36.694041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751028+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:36.751028+0000","last_peered":"2024-04-25T16:27:36.751028+0000","last_clean":"2024-04-25T16:27:36.751028+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:36.751028+0000","last_undegraded":"2024-04-25T16:27:36.751028+0000","last_fullsized":"2024-04-25T16:27:36.751028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699047+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:36.699047+0000","last_peered":"2024-04-25T16:27:36.699047+0000","last_clean":"2024-04-25T16:27:36.699047+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:36.699047+0000","last_undegraded":"2024-04-25T16:27:36.699047+0000","last_fullsized":"2024-04-25T16:27:36.699047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699390+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:36.699390+0000","last_peered":"2024-04-25T16:27:36.699390+0000","last_clean":"2024-04-25T16:27:36.699390+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:36.699390+0000","last_undegraded":"2024-04-25T16:27:36.699390+0000","last_fullsized":"2024-04-25T16:27:36.699390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.703786+0000","last_change":"2024-04-25T16:27:36.703786+0000","last_active":"2024-04-25T16:27:36.703786+0000","last_peered":"2024-04-25T16:27:36.703786+0000","last_clean":"2024-04-25T16:27:36.703786+0000","last_became_active":"2024-04-25T16:27:36.703485+0000","last_became_peered":"2024-04-25T16:27:36.703485+0000","last_unstale":"2024-04-25T16:27:36.703786+0000","last_undegraded":"2024-04-25T16:27:36.703786+0000","last_fullsized":"2024-04-25T16:27:36.703786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:51.980383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694033+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:36.694033+0000","last_peered":"2024-04-25T16:27:36.694033+0000","last_clean":"2024-04-25T16:27:36.694033+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:36.694033+0000","last_undegraded":"2024-04-25T16:27:36.694033+0000","last_fullsized":"2024-04-25T16:27:36.694033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697810+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:36.697810+0000","last_peered":"2024-04-25T16:27:36.697810+0000","last_clean":"2024-04-25T16:27:36.697810+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:36.697810+0000","last_undegraded":"2024-04-25T16:27:36.697810+0000","last_fullsized":"2024-04-25T16:27:36.697810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764249+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:36.764249+0000","last_peered":"2024-04-25T16:27:36.764249+0000","last_clean":"2024-04-25T16:27:36.764249+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:36.764249+0000","last_undegraded":"2024-04-25T16:27:36.764249+0000","last_fullsized":"2024-04-25T16:27:36.764249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697835+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:36.697835+0000","last_peered":"2024-04-25T16:27:36.697835+0000","last_clean":"2024-04-25T16:27:36.697835+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:36.697835+0000","last_undegraded":"2024-04-25T16:27:36.697835+0000","last_fullsized":"2024-04-25T16:27:36.697835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764462+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:36.764462+0000","last_peered":"2024-04-25T16:27:36.764462+0000","last_clean":"2024-04-25T16:27:36.764462+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:36.764462+0000","last_undegraded":"2024-04-25T16:27:36.764462+0000","last_fullsized":"2024-04-25T16:27:36.764462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697522+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:36.697522+0000","last_peered":"2024-04-25T16:27:36.697522+0000","last_clean":"2024-04-25T16:27:36.697522+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:36.697522+0000","last_undegraded":"2024-04-25T16:27:36.697522+0000","last_fullsized":"2024-04-25T16:27:36.697522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697310+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:36.697310+0000","last_peered":"2024-04-25T16:27:36.697310+0000","last_clean":"2024-04-25T16:27:36.697310+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:36.697310+0000","last_undegraded":"2024-04-25T16:27:36.697310+0000","last_fullsized":"2024-04-25T16:27:36.697310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698883+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:36.698883+0000","last_peered":"2024-04-25T16:27:36.698883+0000","last_clean":"2024-04-25T16:27:36.698883+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:36.698883+0000","last_undegraded":"2024-04-25T16:27:36.698883+0000","last_fullsized":"2024-04-25T16:27:36.698883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15331,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":28232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":708870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":132453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":118046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":307200,"data_stored":130023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":117920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":118080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":892928,"data_stored":708926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":12103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:52.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:53.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.034+0000 7f0f65642640 1 Processor -- start 2024-04-25T16:27:53.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.034+0000 7f0f65642640 1 -- start start 2024-04-25T16:27:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f65642640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f6006dc90 0x7f0f600d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f65642640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f600b9b10 0x7f0f600d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f65642640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f600d0870 con 0x7f0f600b5670 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f65642640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0f60160f70 con 0x7f0f6006dc90 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f65642640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0f600b9f20 con 0x7f0f600b9b10 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f600b9b10 0x7f0f600d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5e7fc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f6006dc90 0x7f0f600d4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5effd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0f600b5670 0x7f0f6006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:53406/0 (socket says 172.21.15.94:53406) 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5effd640 1 -- 172.21.15.94:0/2265336836 learned_addr learned my addr 172.21.15.94:0/2265336836 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5dffb640 1 -- 172.21.15.94:0/2265336836 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 331393873 0 0) 0x7f0f600d0870 con 0x7f0f600b5670 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5dffb640 1 -- 172.21.15.94:0/2265336836 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f48003520 con 0x7f0f600b5670 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5f7fe640 1 -- 172.21.15.94:0/2265336836 >> v2:172.21.15.138:3300/0 conn(0x7f0f6006dc90 msgr2=0x7f0f600d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5f7fe640 1 --2- 172.21.15.94:0/2265336836 >> v2:172.21.15.138:3300/0 conn(0x7f0f6006dc90 0x7f0f600d4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5f7fe640 1 -- 172.21.15.94:0/2265336836 >> v1:172.21.15.138:6789/0 conn(0x7f0f600b5670 legacy=0x7f0f6006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5f7fe640 1 -- 172.21.15.94:0/2265336836 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f601651c0 con 0x7f0f600b9b10 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.036+0000 7f0f5e7fc640 1 --2- 172.21.15.94:0/2265336836 >> v2:172.21.15.138:3300/0 conn(0x7f0f6006dc90 0x7f0f600d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.037+0000 7f0f5f7fe640 1 --2- 172.21.15.94:0/2265336836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f600b9b10 0x7f0f600d1310 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097a0 tx=0x7f0f5802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=545717bcf253f0be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.037+0000 7f0f5dffb640 1 -- 172.21.15.94:0/2265336836 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58037070 con 0x7f0f600b9b10 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.037+0000 7f0f5dffb640 1 -- 172.21.15.94:0/2265336836 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f58032440 con 0x7f0f600b9b10 2024-04-25T16:27:53.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.037+0000 7f0f5dffb640 1 -- 172.21.15.94:0/2265336836 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58032810 con 0x7f0f600b9b10 2024-04-25T16:27:53.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 -- 172.21.15.94:0/2265336836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f600b9b10 msgr2=0x7f0f600d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 --2- 172.21.15.94:0/2265336836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f600b9b10 0x7f0f600d1310 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097a0 tx=0x7f0f5802a180 comp rx=0 tx=0).stop 2024-04-25T16:27:53.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 -- 172.21.15.94:0/2265336836 shutdown_connections 2024-04-25T16:27:53.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 --2- 172.21.15.94:0/2265336836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f600b9b10 0x7f0f600d1310 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 --2- 172.21.15.94:0/2265336836 >> v2:172.21.15.138:3300/0 conn(0x7f0f6006dc90 0x7f0f600d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 -- 172.21.15.94:0/2265336836 >> 172.21.15.94:0/2265336836 conn(0x7f0f600b7c70 msgr2=0x7f0f600b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 -- 172.21.15.94:0/2265336836 shutdown_connections 2024-04-25T16:27:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.038+0000 7f0f65642640 1 -- 172.21.15.94:0/2265336836 wait complete. 2024-04-25T16:27:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.040+0000 7f0f65642640 1 Processor -- start 2024-04-25T16:27:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.040+0000 7f0f65642640 1 -- start start 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f65642640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f600b5670 0x7f0f600cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f65642640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 0x7f0f6006c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f65642640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f60154800 0x7f0f6006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f65642640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f601654a0 con 0x7f0f600b9b10 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f65642640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f600d0df0 con 0x7f0f600b5670 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f65642640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0f600bbd50 con 0x7f0f60154800 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f5e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 0x7f0f6006c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f5e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 0x7f0f6006c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42512/0 (socket says 172.21.15.94:42512) 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f5e7fc640 1 -- 172.21.15.94:0/3246946566 learned_addr learned my addr 172.21.15.94:0/3246946566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:53.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f5effd640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f600b5670 0x7f0f600cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.041+0000 7f0f5f7fe640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f60154800 0x7f0f6006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f5e7fc640 1 -- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f60154800 msgr2=0x7f0f6006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f5e7fc640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f60154800 0x7f0f6006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f5e7fc640 1 -- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f600b5670 msgr2=0x7f0f600cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f5e7fc640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f600b5670 0x7f0f600cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f5e7fc640 1 -- 172.21.15.94:0/3246946566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f600cef80 con 0x7f0f600b9b10 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f5e7fc640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 0x7f0f6006c810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e960 tx=0x7f0f5400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54017070 con 0x7f0f600b9b10 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5400c9a0 con 0x7f0f600b9b10 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f601f63d0 con 0x7f0f600b9b10 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400cd70 con 0x7f0f600b9b10 2024-04-25T16:27:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.042+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f601f6b30 con 0x7f0f600b9b10 2024-04-25T16:27:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.043+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f601f65a0 con 0x7f0f600b9b10 2024-04-25T16:27:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.044+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f5401a070 con 0x7f0f600b9b10 2024-04-25T16:27:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.044+0000 7f0f477fe640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0f4c0744e0 0x7f0f4c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.044+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0f54095790 con 0x7f0f600b9b10 2024-04-25T16:27:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.044+0000 7f0f5effd640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0f4c0744e0 0x7f0f4c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.045+0000 7f0f5effd640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0f4c0744e0 0x7f0f4c0769f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005c00 tx=0x7f0f50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.049+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f5405f4d0 con 0x7f0f600b9b10 2024-04-25T16:27:53.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.220+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f600cbce0 con 0x7f0f4c0744e0 2024-04-25T16:27:53.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.228+0000 7f0f477fe640 1 -- 172.21.15.94:0/3246946566 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f0f600cbce0 con 0x7f0f4c0744e0 2024-04-25T16:27:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0f4c0744e0 msgr2=0x7f0f4c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0f4c0744e0 0x7f0f4c0769f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005c00 tx=0x7f0f50009040 comp rx=0 tx=0).stop 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 msgr2=0x7f0f6006c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 0x7f0f6006c810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e960 tx=0x7f0f5400ee20 comp rx=0 tx=0).stop 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 shutdown_connections 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f0f4c0744e0 0x7f0f4c0769f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f60154800 0x7f0f6006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f600b9b10 0x7f0f6006c810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 --2- 172.21.15.94:0/3246946566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f600b5670 0x7f0f600cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 >> 172.21.15.94:0/3246946566 conn(0x7f0f600b7c70 msgr2=0x7f0f6006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 shutdown_connections 2024-04-25T16:27:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.231+0000 7f0f65642640 1 -- 172.21.15.94:0/3246946566 wait complete. 2024-04-25T16:27:53.243 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-04-25T16:27:51.309947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605699,"num_objects":25,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":166,"num_write_kb":1665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236276,"kb_used_data":3484,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738444,"statfs":{"total":773094113280,"available":772852166656,"internally_reserved":0,"allocated":3567616,"data_stored":2152328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10179,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699051+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:36.699051+0000","last_peered":"2024-04-25T16:27:36.699051+0000","last_clean":"2024-04-25T16:27:36.699051+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:36.699051+0000","last_undegraded":"2024-04-25T16:27:36.699051+0000","last_fullsized":"2024-04-25T16:27:36.699051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694756+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:36.694756+0000","last_peered":"2024-04-25T16:27:36.694756+0000","last_clean":"2024-04-25T16:27:36.694756+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:36.694756+0000","last_undegraded":"2024-04-25T16:27:36.694756+0000","last_fullsized":"2024-04-25T16:27:36.694756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693795+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:36.693795+0000","last_peered":"2024-04-25T16:27:36.693795+0000","last_clean":"2024-04-25T16:27:36.693795+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:36.693795+0000","last_undegraded":"2024-04-25T16:27:36.693795+0000","last_fullsized":"2024-04-25T16:27:36.693795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694849+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:36.694849+0000","last_peered":"2024-04-25T16:27:36.694849+0000","last_clean":"2024-04-25T16:27:36.694849+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:36.694849+0000","last_undegraded":"2024-04-25T16:27:36.694849+0000","last_fullsized":"2024-04-25T16:27:36.694849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697915+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:36.697915+0000","last_peered":"2024-04-25T16:27:36.697915+0000","last_clean":"2024-04-25T16:27:36.697915+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:36.697915+0000","last_undegraded":"2024-04-25T16:27:36.697915+0000","last_fullsized":"2024-04-25T16:27:36.697915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750192+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:36.750192+0000","last_peered":"2024-04-25T16:27:36.750192+0000","last_clean":"2024-04-25T16:27:36.750192+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:36.750192+0000","last_undegraded":"2024-04-25T16:27:36.750192+0000","last_fullsized":"2024-04-25T16:27:36.750192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699035+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:36.699035+0000","last_peered":"2024-04-25T16:27:36.699035+0000","last_clean":"2024-04-25T16:27:36.699035+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:36.699035+0000","last_undegraded":"2024-04-25T16:27:36.699035+0000","last_fullsized":"2024-04-25T16:27:36.699035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764530+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:36.764530+0000","last_peered":"2024-04-25T16:27:36.764530+0000","last_clean":"2024-04-25T16:27:36.764530+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:36.764530+0000","last_undegraded":"2024-04-25T16:27:36.764530+0000","last_fullsized":"2024-04-25T16:27:36.764530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750366+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:36.750366+0000","last_peered":"2024-04-25T16:27:36.750366+0000","last_clean":"2024-04-25T16:27:36.750366+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:36.750366+0000","last_undegraded":"2024-04-25T16:27:36.750366+0000","last_fullsized":"2024-04-25T16:27:36.750366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697999+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:36.697999+0000","last_peered":"2024-04-25T16:27:36.697999+0000","last_clean":"2024-04-25T16:27:36.697999+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:36.697999+0000","last_undegraded":"2024-04-25T16:27:36.697999+0000","last_fullsized":"2024-04-25T16:27:36.697999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764669+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:36.764669+0000","last_peered":"2024-04-25T16:27:36.764669+0000","last_clean":"2024-04-25T16:27:36.764669+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:36.764669+0000","last_undegraded":"2024-04-25T16:27:36.764669+0000","last_fullsized":"2024-04-25T16:27:36.764669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750279+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:36.750279+0000","last_peered":"2024-04-25T16:27:36.750279+0000","last_clean":"2024-04-25T16:27:36.750279+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:36.750279+0000","last_undegraded":"2024-04-25T16:27:36.750279+0000","last_fullsized":"2024-04-25T16:27:36.750279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697286+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:36.697286+0000","last_peered":"2024-04-25T16:27:36.697286+0000","last_clean":"2024-04-25T16:27:36.697286+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:36.697286+0000","last_undegraded":"2024-04-25T16:27:36.697286+0000","last_fullsized":"2024-04-25T16:27:36.697286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698555+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:36.698555+0000","last_peered":"2024-04-25T16:27:36.698555+0000","last_clean":"2024-04-25T16:27:36.698555+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:36.698555+0000","last_undegraded":"2024-04-25T16:27:36.698555+0000","last_fullsized":"2024-04-25T16:27:36.698555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750951+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:36.750951+0000","last_peered":"2024-04-25T16:27:36.750951+0000","last_clean":"2024-04-25T16:27:36.750951+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:36.750951+0000","last_undegraded":"2024-04-25T16:27:36.750951+0000","last_fullsized":"2024-04-25T16:27:36.750951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694265+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:36.694265+0000","last_peered":"2024-04-25T16:27:36.694265+0000","last_clean":"2024-04-25T16:27:36.694265+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:36.694265+0000","last_undegraded":"2024-04-25T16:27:36.694265+0000","last_fullsized":"2024-04-25T16:27:36.694265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697095+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:36.697095+0000","last_peered":"2024-04-25T16:27:36.697095+0000","last_clean":"2024-04-25T16:27:36.697095+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:36.697095+0000","last_undegraded":"2024-04-25T16:27:36.697095+0000","last_fullsized":"2024-04-25T16:27:36.697095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750759+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:36.750759+0000","last_peered":"2024-04-25T16:27:36.750759+0000","last_clean":"2024-04-25T16:27:36.750759+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:36.750759+0000","last_undegraded":"2024-04-25T16:27:36.750759+0000","last_fullsized":"2024-04-25T16:27:36.750759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694142+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:36.694142+0000","last_peered":"2024-04-25T16:27:36.694142+0000","last_clean":"2024-04-25T16:27:36.694142+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:36.694142+0000","last_undegraded":"2024-04-25T16:27:36.694142+0000","last_fullsized":"2024-04-25T16:27:36.694142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696965+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:36.696965+0000","last_peered":"2024-04-25T16:27:36.696965+0000","last_clean":"2024-04-25T16:27:36.696965+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:36.696965+0000","last_undegraded":"2024-04-25T16:27:36.696965+0000","last_fullsized":"2024-04-25T16:27:36.696965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.752445+0000","last_change":"2024-04-25T16:27:36.751906+0000","last_active":"2024-04-25T16:27:36.752445+0000","last_peered":"2024-04-25T16:27:36.752445+0000","last_clean":"2024-04-25T16:27:36.752445+0000","last_became_active":"2024-04-25T16:27:36.751463+0000","last_became_peered":"2024-04-25T16:27:36.751463+0000","last_unstale":"2024-04-25T16:27:36.752445+0000","last_undegraded":"2024-04-25T16:27:36.752445+0000","last_fullsized":"2024-04-25T16:27:36.752445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750568+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:36.750568+0000","last_peered":"2024-04-25T16:27:36.750568+0000","last_clean":"2024-04-25T16:27:36.750568+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:36.750568+0000","last_undegraded":"2024-04-25T16:27:36.750568+0000","last_fullsized":"2024-04-25T16:27:36.750568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697515+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:36.697515+0000","last_peered":"2024-04-25T16:27:36.697515+0000","last_clean":"2024-04-25T16:27:36.697515+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:36.697515+0000","last_undegraded":"2024-04-25T16:27:36.697515+0000","last_fullsized":"2024-04-25T16:27:36.697515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750847+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:36.750847+0000","last_peered":"2024-04-25T16:27:36.750847+0000","last_clean":"2024-04-25T16:27:36.750847+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:36.750847+0000","last_undegraded":"2024-04-25T16:27:36.750847+0000","last_fullsized":"2024-04-25T16:27:36.750847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698515+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:36.698515+0000","last_peered":"2024-04-25T16:27:36.698515+0000","last_clean":"2024-04-25T16:27:36.698515+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:36.698515+0000","last_undegraded":"2024-04-25T16:27:36.698515+0000","last_fullsized":"2024-04-25T16:27:36.698515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.700039+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:36.700039+0000","last_peered":"2024-04-25T16:27:36.700039+0000","last_clean":"2024-04-25T16:27:36.700039+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:36.700039+0000","last_undegraded":"2024-04-25T16:27:36.700039+0000","last_fullsized":"2024-04-25T16:27:36.700039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750332+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:36.750332+0000","last_peered":"2024-04-25T16:27:36.750332+0000","last_clean":"2024-04-25T16:27:36.750332+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:36.750332+0000","last_undegraded":"2024-04-25T16:27:36.750332+0000","last_fullsized":"2024-04-25T16:27:36.750332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.695158+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:36.695158+0000","last_peered":"2024-04-25T16:27:36.695158+0000","last_clean":"2024-04-25T16:27:36.695158+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:36.695158+0000","last_undegraded":"2024-04-25T16:27:36.695158+0000","last_fullsized":"2024-04-25T16:27:36.695158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:41.552216+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:41.552216+0000","last_peered":"2024-04-25T16:27:41.552216+0000","last_clean":"2024-04-25T16:27:41.552216+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:41.552216+0000","last_undegraded":"2024-04-25T16:27:41.552216+0000","last_fullsized":"2024-04-25T16:27:41.552216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693728+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:36.693728+0000","last_peered":"2024-04-25T16:27:36.693728+0000","last_clean":"2024-04-25T16:27:36.693728+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:36.693728+0000","last_undegraded":"2024-04-25T16:27:36.693728+0000","last_fullsized":"2024-04-25T16:27:36.693728+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697033+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:36.697033+0000","last_peered":"2024-04-25T16:27:36.697033+0000","last_clean":"2024-04-25T16:27:36.697033+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:36.697033+0000","last_undegraded":"2024-04-25T16:27:36.697033+0000","last_fullsized":"2024-04-25T16:27:36.697033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698561+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:36.698561+0000","last_peered":"2024-04-25T16:27:36.698561+0000","last_clean":"2024-04-25T16:27:36.698561+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:36.698561+0000","last_undegraded":"2024-04-25T16:27:36.698561+0000","last_fullsized":"2024-04-25T16:27:36.698561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698300+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:36.698300+0000","last_peered":"2024-04-25T16:27:36.698300+0000","last_clean":"2024-04-25T16:27:36.698300+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:36.698300+0000","last_undegraded":"2024-04-25T16:27:36.698300+0000","last_fullsized":"2024-04-25T16:27:36.698300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764769+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:36.764769+0000","last_peered":"2024-04-25T16:27:36.764769+0000","last_clean":"2024-04-25T16:27:36.764769+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:36.764769+0000","last_undegraded":"2024-04-25T16:27:36.764769+0000","last_fullsized":"2024-04-25T16:27:36.764769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698328+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:36.698328+0000","last_peered":"2024-04-25T16:27:36.698328+0000","last_clean":"2024-04-25T16:27:36.698328+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:36.698328+0000","last_undegraded":"2024-04-25T16:27:36.698328+0000","last_fullsized":"2024-04-25T16:27:36.698328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696738+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:36.696738+0000","last_peered":"2024-04-25T16:27:36.696738+0000","last_clean":"2024-04-25T16:27:36.696738+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:36.696738+0000","last_undegraded":"2024-04-25T16:27:36.696738+0000","last_fullsized":"2024-04-25T16:27:36.696738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750724+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:36.750724+0000","last_peered":"2024-04-25T16:27:36.750724+0000","last_clean":"2024-04-25T16:27:36.750724+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:36.750724+0000","last_undegraded":"2024-04-25T16:27:36.750724+0000","last_fullsized":"2024-04-25T16:27:36.750724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764844+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:36.764844+0000","last_peered":"2024-04-25T16:27:36.764844+0000","last_clean":"2024-04-25T16:27:36.764844+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:36.764844+0000","last_undegraded":"2024-04-25T16:27:36.764844+0000","last_fullsized":"2024-04-25T16:27:36.764844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696930+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:36.696930+0000","last_peered":"2024-04-25T16:27:36.696930+0000","last_clean":"2024-04-25T16:27:36.696930+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:36.696930+0000","last_undegraded":"2024-04-25T16:27:36.696930+0000","last_fullsized":"2024-04-25T16:27:36.696930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698136+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:36.698136+0000","last_peered":"2024-04-25T16:27:36.698136+0000","last_clean":"2024-04-25T16:27:36.698136+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:36.698136+0000","last_undegraded":"2024-04-25T16:27:36.698136+0000","last_fullsized":"2024-04-25T16:27:36.698136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697778+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:36.697778+0000","last_peered":"2024-04-25T16:27:36.697778+0000","last_clean":"2024-04-25T16:27:36.697778+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:36.697778+0000","last_undegraded":"2024-04-25T16:27:36.697778+0000","last_fullsized":"2024-04-25T16:27:36.697778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697213+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:36.697213+0000","last_peered":"2024-04-25T16:27:36.697213+0000","last_clean":"2024-04-25T16:27:36.697213+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:36.697213+0000","last_undegraded":"2024-04-25T16:27:36.697213+0000","last_fullsized":"2024-04-25T16:27:36.697213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697266+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:36.697266+0000","last_peered":"2024-04-25T16:27:36.697266+0000","last_clean":"2024-04-25T16:27:36.697266+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:36.697266+0000","last_undegraded":"2024-04-25T16:27:36.697266+0000","last_fullsized":"2024-04-25T16:27:36.697266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697357+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:36.697357+0000","last_peered":"2024-04-25T16:27:36.697357+0000","last_clean":"2024-04-25T16:27:36.697357+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:36.697357+0000","last_undegraded":"2024-04-25T16:27:36.697357+0000","last_fullsized":"2024-04-25T16:27:36.697357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698254+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:36.698254+0000","last_peered":"2024-04-25T16:27:36.698254+0000","last_clean":"2024-04-25T16:27:36.698254+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:36.698254+0000","last_undegraded":"2024-04-25T16:27:36.698254+0000","last_fullsized":"2024-04-25T16:27:36.698254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694153+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:36.694153+0000","last_peered":"2024-04-25T16:27:36.694153+0000","last_clean":"2024-04-25T16:27:36.694153+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:36.694153+0000","last_undegraded":"2024-04-25T16:27:36.694153+0000","last_fullsized":"2024-04-25T16:27:36.694153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764525+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:36.764525+0000","last_peered":"2024-04-25T16:27:36.764525+0000","last_clean":"2024-04-25T16:27:36.764525+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:36.764525+0000","last_undegraded":"2024-04-25T16:27:36.764525+0000","last_fullsized":"2024-04-25T16:27:36.764525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697495+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:36.697495+0000","last_peered":"2024-04-25T16:27:36.697495+0000","last_clean":"2024-04-25T16:27:36.697495+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:36.697495+0000","last_undegraded":"2024-04-25T16:27:36.697495+0000","last_fullsized":"2024-04-25T16:27:36.697495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698810+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:36.698810+0000","last_peered":"2024-04-25T16:27:36.698810+0000","last_clean":"2024-04-25T16:27:36.698810+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:36.698810+0000","last_undegraded":"2024-04-25T16:27:36.698810+0000","last_fullsized":"2024-04-25T16:27:36.698810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698193+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:36.698193+0000","last_peered":"2024-04-25T16:27:36.698193+0000","last_clean":"2024-04-25T16:27:36.698193+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:36.698193+0000","last_undegraded":"2024-04-25T16:27:36.698193+0000","last_fullsized":"2024-04-25T16:27:36.698193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697778+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:36.697778+0000","last_peered":"2024-04-25T16:27:36.697778+0000","last_clean":"2024-04-25T16:27:36.697778+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:36.697778+0000","last_undegraded":"2024-04-25T16:27:36.697778+0000","last_fullsized":"2024-04-25T16:27:36.697778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697238+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:36.697238+0000","last_peered":"2024-04-25T16:27:36.697238+0000","last_clean":"2024-04-25T16:27:36.697238+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:36.697238+0000","last_undegraded":"2024-04-25T16:27:36.697238+0000","last_fullsized":"2024-04-25T16:27:36.697238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764579+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:36.764579+0000","last_peered":"2024-04-25T16:27:36.764579+0000","last_clean":"2024-04-25T16:27:36.764579+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:36.764579+0000","last_undegraded":"2024-04-25T16:27:36.764579+0000","last_fullsized":"2024-04-25T16:27:36.764579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697603+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:36.697603+0000","last_peered":"2024-04-25T16:27:36.697603+0000","last_clean":"2024-04-25T16:27:36.697603+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:36.697603+0000","last_undegraded":"2024-04-25T16:27:36.697603+0000","last_fullsized":"2024-04-25T16:27:36.697603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698509+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:36.698509+0000","last_peered":"2024-04-25T16:27:36.698509+0000","last_clean":"2024-04-25T16:27:36.698509+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:36.698509+0000","last_undegraded":"2024-04-25T16:27:36.698509+0000","last_fullsized":"2024-04-25T16:27:36.698509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699318+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:36.699318+0000","last_peered":"2024-04-25T16:27:36.699318+0000","last_clean":"2024-04-25T16:27:36.699318+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:36.699318+0000","last_undegraded":"2024-04-25T16:27:36.699318+0000","last_fullsized":"2024-04-25T16:27:36.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697295+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:36.697295+0000","last_peered":"2024-04-25T16:27:36.697295+0000","last_clean":"2024-04-25T16:27:36.697295+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:36.697295+0000","last_undegraded":"2024-04-25T16:27:36.697295+0000","last_fullsized":"2024-04-25T16:27:36.697295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.763861+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:36.763861+0000","last_peered":"2024-04-25T16:27:36.763861+0000","last_clean":"2024-04-25T16:27:36.763861+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:36.763861+0000","last_undegraded":"2024-04-25T16:27:36.763861+0000","last_fullsized":"2024-04-25T16:27:36.763861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698846+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:36.698846+0000","last_peered":"2024-04-25T16:27:36.698846+0000","last_clean":"2024-04-25T16:27:36.698846+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:36.698846+0000","last_undegraded":"2024-04-25T16:27:36.698846+0000","last_fullsized":"2024-04-25T16:27:36.698846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751170+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:36.751170+0000","last_peered":"2024-04-25T16:27:36.751170+0000","last_clean":"2024-04-25T16:27:36.751170+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:36.751170+0000","last_undegraded":"2024-04-25T16:27:36.751170+0000","last_fullsized":"2024-04-25T16:27:36.751170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750149+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:36.750149+0000","last_peered":"2024-04-25T16:27:36.750149+0000","last_clean":"2024-04-25T16:27:36.750149+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:36.750149+0000","last_undegraded":"2024-04-25T16:27:36.750149+0000","last_fullsized":"2024-04-25T16:27:36.750149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698175+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:36.698175+0000","last_peered":"2024-04-25T16:27:36.698175+0000","last_clean":"2024-04-25T16:27:36.698175+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:36.698175+0000","last_undegraded":"2024-04-25T16:27:36.698175+0000","last_fullsized":"2024-04-25T16:27:36.698175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697542+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:36.697542+0000","last_peered":"2024-04-25T16:27:36.697542+0000","last_clean":"2024-04-25T16:27:36.697542+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:36.697542+0000","last_undegraded":"2024-04-25T16:27:36.697542+0000","last_fullsized":"2024-04-25T16:27:36.697542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698129+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:36.698129+0000","last_peered":"2024-04-25T16:27:36.698129+0000","last_clean":"2024-04-25T16:27:36.698129+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:36.698129+0000","last_undegraded":"2024-04-25T16:27:36.698129+0000","last_fullsized":"2024-04-25T16:27:36.698129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697530+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:36.697530+0000","last_peered":"2024-04-25T16:27:36.697530+0000","last_clean":"2024-04-25T16:27:36.697530+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:36.697530+0000","last_undegraded":"2024-04-25T16:27:36.697530+0000","last_fullsized":"2024-04-25T16:27:36.697530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697833+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:36.697833+0000","last_peered":"2024-04-25T16:27:36.697833+0000","last_clean":"2024-04-25T16:27:36.697833+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:36.697833+0000","last_undegraded":"2024-04-25T16:27:36.697833+0000","last_fullsized":"2024-04-25T16:27:36.697833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694937+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:36.694937+0000","last_peered":"2024-04-25T16:27:36.694937+0000","last_clean":"2024-04-25T16:27:36.694937+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:36.694937+0000","last_undegraded":"2024-04-25T16:27:36.694937+0000","last_fullsized":"2024-04-25T16:27:36.694937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698423+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:36.698423+0000","last_peered":"2024-04-25T16:27:36.698423+0000","last_clean":"2024-04-25T16:27:36.698423+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:36.698423+0000","last_undegraded":"2024-04-25T16:27:36.698423+0000","last_fullsized":"2024-04-25T16:27:36.698423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764677+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:36.764677+0000","last_peered":"2024-04-25T16:27:36.764677+0000","last_clean":"2024-04-25T16:27:36.764677+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:36.764677+0000","last_undegraded":"2024-04-25T16:27:36.764677+0000","last_fullsized":"2024-04-25T16:27:36.764677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694274+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:36.694274+0000","last_peered":"2024-04-25T16:27:36.694274+0000","last_clean":"2024-04-25T16:27:36.694274+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:36.694274+0000","last_undegraded":"2024-04-25T16:27:36.694274+0000","last_fullsized":"2024-04-25T16:27:36.694274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698335+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:36.698335+0000","last_peered":"2024-04-25T16:27:36.698335+0000","last_clean":"2024-04-25T16:27:36.698335+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:36.698335+0000","last_undegraded":"2024-04-25T16:27:36.698335+0000","last_fullsized":"2024-04-25T16:27:36.698335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698767+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:36.698767+0000","last_peered":"2024-04-25T16:27:36.698767+0000","last_clean":"2024-04-25T16:27:36.698767+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:36.698767+0000","last_undegraded":"2024-04-25T16:27:36.698767+0000","last_fullsized":"2024-04-25T16:27:36.698767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764506+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:36.764506+0000","last_peered":"2024-04-25T16:27:36.764506+0000","last_clean":"2024-04-25T16:27:36.764506+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:36.764506+0000","last_undegraded":"2024-04-25T16:27:36.764506+0000","last_fullsized":"2024-04-25T16:27:36.764506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698073+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:36.698073+0000","last_peered":"2024-04-25T16:27:36.698073+0000","last_clean":"2024-04-25T16:27:36.698073+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:36.698073+0000","last_undegraded":"2024-04-25T16:27:36.698073+0000","last_fullsized":"2024-04-25T16:27:36.698073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'21","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:47.513651+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:47.513651+0000","last_peered":"2024-04-25T16:27:47.513651+0000","last_clean":"2024-04-25T16:27:47.513651+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:47.513651+0000","last_undegraded":"2024-04-25T16:27:47.513651+0000","last_fullsized":"2024-04-25T16:27:47.513651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696855+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:36.696855+0000","last_peered":"2024-04-25T16:27:36.696855+0000","last_clean":"2024-04-25T16:27:36.696855+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:36.696855+0000","last_undegraded":"2024-04-25T16:27:36.696855+0000","last_fullsized":"2024-04-25T16:27:36.696855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.701265+0000","last_change":"2024-04-25T16:27:36.700437+0000","last_active":"2024-04-25T16:27:36.701265+0000","last_peered":"2024-04-25T16:27:36.701265+0000","last_clean":"2024-04-25T16:27:36.701265+0000","last_became_active":"2024-04-25T16:27:36.700105+0000","last_became_peered":"2024-04-25T16:27:36.700105+0000","last_unstale":"2024-04-25T16:27:36.701265+0000","last_undegraded":"2024-04-25T16:27:36.701265+0000","last_fullsized":"2024-04-25T16:27:36.701265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697453+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:36.697453+0000","last_peered":"2024-04-25T16:27:36.697453+0000","last_clean":"2024-04-25T16:27:36.697453+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:36.697453+0000","last_undegraded":"2024-04-25T16:27:36.697453+0000","last_fullsized":"2024-04-25T16:27:36.697453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750587+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:36.750587+0000","last_peered":"2024-04-25T16:27:36.750587+0000","last_clean":"2024-04-25T16:27:36.750587+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:36.750587+0000","last_undegraded":"2024-04-25T16:27:36.750587+0000","last_fullsized":"2024-04-25T16:27:36.750587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698936+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:36.698936+0000","last_peered":"2024-04-25T16:27:36.698936+0000","last_clean":"2024-04-25T16:27:36.698936+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:36.698936+0000","last_undegraded":"2024-04-25T16:27:36.698936+0000","last_fullsized":"2024-04-25T16:27:36.698936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764563+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:36.764563+0000","last_peered":"2024-04-25T16:27:36.764563+0000","last_clean":"2024-04-25T16:27:36.764563+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:36.764563+0000","last_undegraded":"2024-04-25T16:27:36.764563+0000","last_fullsized":"2024-04-25T16:27:36.764563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:41.196000+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:41.196000+0000","last_peered":"2024-04-25T16:27:41.196000+0000","last_clean":"2024-04-25T16:27:41.196000+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:41.196000+0000","last_undegraded":"2024-04-25T16:27:41.196000+0000","last_fullsized":"2024-04-25T16:27:41.196000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764463+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:36.764463+0000","last_peered":"2024-04-25T16:27:36.764463+0000","last_clean":"2024-04-25T16:27:36.764463+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:36.764463+0000","last_undegraded":"2024-04-25T16:27:36.764463+0000","last_fullsized":"2024-04-25T16:27:36.764463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699948+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:36.699948+0000","last_peered":"2024-04-25T16:27:36.699948+0000","last_clean":"2024-04-25T16:27:36.699948+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:36.699948+0000","last_undegraded":"2024-04-25T16:27:36.699948+0000","last_fullsized":"2024-04-25T16:27:36.699948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693864+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:36.693864+0000","last_peered":"2024-04-25T16:27:36.693864+0000","last_clean":"2024-04-25T16:27:36.693864+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:36.693864+0000","last_undegraded":"2024-04-25T16:27:36.693864+0000","last_fullsized":"2024-04-25T16:27:36.693864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.702563+0000","last_change":"2024-04-25T16:27:36.702563+0000","last_active":"2024-04-25T16:27:36.702563+0000","last_peered":"2024-04-25T16:27:36.702563+0000","last_clean":"2024-04-25T16:27:36.702563+0000","last_became_active":"2024-04-25T16:27:36.702077+0000","last_became_peered":"2024-04-25T16:27:36.702077+0000","last_unstale":"2024-04-25T16:27:36.702563+0000","last_undegraded":"2024-04-25T16:27:36.702563+0000","last_fullsized":"2024-04-25T16:27:36.702563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:48.724249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,2],"acting":[0,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.701750+0000","last_change":"2024-04-25T16:27:36.701750+0000","last_active":"2024-04-25T16:27:36.701750+0000","last_peered":"2024-04-25T16:27:36.701750+0000","last_clean":"2024-04-25T16:27:36.701750+0000","last_became_active":"2024-04-25T16:27:36.701427+0000","last_became_peered":"2024-04-25T16:27:36.701427+0000","last_unstale":"2024-04-25T16:27:36.701750+0000","last_undegraded":"2024-04-25T16:27:36.701750+0000","last_fullsized":"2024-04-25T16:27:36.701750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:44.695661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764137+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:36.764137+0000","last_peered":"2024-04-25T16:27:36.764137+0000","last_clean":"2024-04-25T16:27:36.764137+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:36.764137+0000","last_undegraded":"2024-04-25T16:27:36.764137+0000","last_fullsized":"2024-04-25T16:27:36.764137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.766198+0000","last_change":"2024-04-25T16:27:36.766198+0000","last_active":"2024-04-25T16:27:36.766198+0000","last_peered":"2024-04-25T16:27:36.766198+0000","last_clean":"2024-04-25T16:27:36.766198+0000","last_became_active":"2024-04-25T16:27:36.765669+0000","last_became_peered":"2024-04-25T16:27:36.765669+0000","last_unstale":"2024-04-25T16:27:36.766198+0000","last_undegraded":"2024-04-25T16:27:36.766198+0000","last_fullsized":"2024-04-25T16:27:36.766198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:30.911634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694021+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:36.694021+0000","last_peered":"2024-04-25T16:27:36.694021+0000","last_clean":"2024-04-25T16:27:36.694021+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:36.694021+0000","last_undegraded":"2024-04-25T16:27:36.694021+0000","last_fullsized":"2024-04-25T16:27:36.694021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694344+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:36.694344+0000","last_peered":"2024-04-25T16:27:36.694344+0000","last_clean":"2024-04-25T16:27:36.694344+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:36.694344+0000","last_undegraded":"2024-04-25T16:27:36.694344+0000","last_fullsized":"2024-04-25T16:27:36.694344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698727+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:36.698727+0000","last_peered":"2024-04-25T16:27:36.698727+0000","last_clean":"2024-04-25T16:27:36.698727+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:36.698727+0000","last_undegraded":"2024-04-25T16:27:36.698727+0000","last_fullsized":"2024-04-25T16:27:36.698727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698703+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:36.698703+0000","last_peered":"2024-04-25T16:27:36.698703+0000","last_clean":"2024-04-25T16:27:36.698703+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:36.698703+0000","last_undegraded":"2024-04-25T16:27:36.698703+0000","last_fullsized":"2024-04-25T16:27:36.698703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697541+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:36.697541+0000","last_peered":"2024-04-25T16:27:36.697541+0000","last_clean":"2024-04-25T16:27:36.697541+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:36.697541+0000","last_undegraded":"2024-04-25T16:27:36.697541+0000","last_fullsized":"2024-04-25T16:27:36.697541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694183+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:36.694183+0000","last_peered":"2024-04-25T16:27:36.694183+0000","last_clean":"2024-04-25T16:27:36.694183+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:36.694183+0000","last_undegraded":"2024-04-25T16:27:36.694183+0000","last_fullsized":"2024-04-25T16:27:36.694183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750593+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:36.750593+0000","last_peered":"2024-04-25T16:27:36.750593+0000","last_clean":"2024-04-25T16:27:36.750593+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:36.750593+0000","last_undegraded":"2024-04-25T16:27:36.750593+0000","last_fullsized":"2024-04-25T16:27:36.750593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694424+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:36.694424+0000","last_peered":"2024-04-25T16:27:36.694424+0000","last_clean":"2024-04-25T16:27:36.694424+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:36.694424+0000","last_undegraded":"2024-04-25T16:27:36.694424+0000","last_fullsized":"2024-04-25T16:27:36.694424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697403+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:36.697403+0000","last_peered":"2024-04-25T16:27:36.697403+0000","last_clean":"2024-04-25T16:27:36.697403+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:36.697403+0000","last_undegraded":"2024-04-25T16:27:36.697403+0000","last_fullsized":"2024-04-25T16:27:36.697403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697750+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:36.697750+0000","last_peered":"2024-04-25T16:27:36.697750+0000","last_clean":"2024-04-25T16:27:36.697750+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:36.697750+0000","last_undegraded":"2024-04-25T16:27:36.697750+0000","last_fullsized":"2024-04-25T16:27:36.697750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.702267+0000","last_change":"2024-04-25T16:27:36.701488+0000","last_active":"2024-04-25T16:27:36.702267+0000","last_peered":"2024-04-25T16:27:36.702267+0000","last_clean":"2024-04-25T16:27:36.702267+0000","last_became_active":"2024-04-25T16:27:36.701156+0000","last_became_peered":"2024-04-25T16:27:36.701156+0000","last_unstale":"2024-04-25T16:27:36.702267+0000","last_undegraded":"2024-04-25T16:27:36.702267+0000","last_fullsized":"2024-04-25T16:27:36.702267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698662+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:36.698662+0000","last_peered":"2024-04-25T16:27:36.698662+0000","last_clean":"2024-04-25T16:27:36.698662+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:36.698662+0000","last_undegraded":"2024-04-25T16:27:36.698662+0000","last_fullsized":"2024-04-25T16:27:36.698662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694391+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:36.694391+0000","last_peered":"2024-04-25T16:27:36.694391+0000","last_clean":"2024-04-25T16:27:36.694391+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:36.694391+0000","last_undegraded":"2024-04-25T16:27:36.694391+0000","last_fullsized":"2024-04-25T16:27:36.694391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696935+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:36.696935+0000","last_peered":"2024-04-25T16:27:36.696935+0000","last_clean":"2024-04-25T16:27:36.696935+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:36.696935+0000","last_undegraded":"2024-04-25T16:27:36.696935+0000","last_fullsized":"2024-04-25T16:27:36.696935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699512+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:36.699512+0000","last_peered":"2024-04-25T16:27:36.699512+0000","last_clean":"2024-04-25T16:27:36.699512+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:36.699512+0000","last_undegraded":"2024-04-25T16:27:36.699512+0000","last_fullsized":"2024-04-25T16:27:36.699512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.695043+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:36.695043+0000","last_peered":"2024-04-25T16:27:36.695043+0000","last_clean":"2024-04-25T16:27:36.695043+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:36.695043+0000","last_undegraded":"2024-04-25T16:27:36.695043+0000","last_fullsized":"2024-04-25T16:27:36.695043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764019+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:36.764019+0000","last_peered":"2024-04-25T16:27:36.764019+0000","last_clean":"2024-04-25T16:27:36.764019+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:36.764019+0000","last_undegraded":"2024-04-25T16:27:36.764019+0000","last_fullsized":"2024-04-25T16:27:36.764019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694199+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:36.694199+0000","last_peered":"2024-04-25T16:27:36.694199+0000","last_clean":"2024-04-25T16:27:36.694199+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:36.694199+0000","last_undegraded":"2024-04-25T16:27:36.694199+0000","last_fullsized":"2024-04-25T16:27:36.694199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764266+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:36.764266+0000","last_peered":"2024-04-25T16:27:36.764266+0000","last_clean":"2024-04-25T16:27:36.764266+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:36.764266+0000","last_undegraded":"2024-04-25T16:27:36.764266+0000","last_fullsized":"2024-04-25T16:27:36.764266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698420+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:36.698420+0000","last_peered":"2024-04-25T16:27:36.698420+0000","last_clean":"2024-04-25T16:27:36.698420+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:36.698420+0000","last_undegraded":"2024-04-25T16:27:36.698420+0000","last_fullsized":"2024-04-25T16:27:36.698420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698766+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:36.698766+0000","last_peered":"2024-04-25T16:27:36.698766+0000","last_clean":"2024-04-25T16:27:36.698766+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:36.698766+0000","last_undegraded":"2024-04-25T16:27:36.698766+0000","last_fullsized":"2024-04-25T16:27:36.698766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698354+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:36.698354+0000","last_peered":"2024-04-25T16:27:36.698354+0000","last_clean":"2024-04-25T16:27:36.698354+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:36.698354+0000","last_undegraded":"2024-04-25T16:27:36.698354+0000","last_fullsized":"2024-04-25T16:27:36.698354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697177+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:36.697177+0000","last_peered":"2024-04-25T16:27:36.697177+0000","last_clean":"2024-04-25T16:27:36.697177+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:36.697177+0000","last_undegraded":"2024-04-25T16:27:36.697177+0000","last_fullsized":"2024-04-25T16:27:36.697177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764875+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:36.764875+0000","last_peered":"2024-04-25T16:27:36.764875+0000","last_clean":"2024-04-25T16:27:36.764875+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:36.764875+0000","last_undegraded":"2024-04-25T16:27:36.764875+0000","last_fullsized":"2024-04-25T16:27:36.764875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764030+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:36.764030+0000","last_peered":"2024-04-25T16:27:36.764030+0000","last_clean":"2024-04-25T16:27:36.764030+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:36.764030+0000","last_undegraded":"2024-04-25T16:27:36.764030+0000","last_fullsized":"2024-04-25T16:27:36.764030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696865+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:36.696865+0000","last_peered":"2024-04-25T16:27:36.696865+0000","last_clean":"2024-04-25T16:27:36.696865+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:36.696865+0000","last_undegraded":"2024-04-25T16:27:36.696865+0000","last_fullsized":"2024-04-25T16:27:36.696865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698226+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:36.698226+0000","last_peered":"2024-04-25T16:27:36.698226+0000","last_clean":"2024-04-25T16:27:36.698226+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:36.698226+0000","last_undegraded":"2024-04-25T16:27:36.698226+0000","last_fullsized":"2024-04-25T16:27:36.698226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694041+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:36.694041+0000","last_peered":"2024-04-25T16:27:36.694041+0000","last_clean":"2024-04-25T16:27:36.694041+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:36.694041+0000","last_undegraded":"2024-04-25T16:27:36.694041+0000","last_fullsized":"2024-04-25T16:27:36.694041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751028+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:36.751028+0000","last_peered":"2024-04-25T16:27:36.751028+0000","last_clean":"2024-04-25T16:27:36.751028+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:36.751028+0000","last_undegraded":"2024-04-25T16:27:36.751028+0000","last_fullsized":"2024-04-25T16:27:36.751028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699047+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:36.699047+0000","last_peered":"2024-04-25T16:27:36.699047+0000","last_clean":"2024-04-25T16:27:36.699047+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:36.699047+0000","last_undegraded":"2024-04-25T16:27:36.699047+0000","last_fullsized":"2024-04-25T16:27:36.699047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699390+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:36.699390+0000","last_peered":"2024-04-25T16:27:36.699390+0000","last_clean":"2024-04-25T16:27:36.699390+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:36.699390+0000","last_undegraded":"2024-04-25T16:27:36.699390+0000","last_fullsized":"2024-04-25T16:27:36.699390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.703786+0000","last_change":"2024-04-25T16:27:36.703786+0000","last_active":"2024-04-25T16:27:36.703786+0000","last_peered":"2024-04-25T16:27:36.703786+0000","last_clean":"2024-04-25T16:27:36.703786+0000","last_became_active":"2024-04-25T16:27:36.703485+0000","last_became_peered":"2024-04-25T16:27:36.703485+0000","last_unstale":"2024-04-25T16:27:36.703786+0000","last_undegraded":"2024-04-25T16:27:36.703786+0000","last_fullsized":"2024-04-25T16:27:36.703786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:51.980383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694033+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:36.694033+0000","last_peered":"2024-04-25T16:27:36.694033+0000","last_clean":"2024-04-25T16:27:36.694033+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:36.694033+0000","last_undegraded":"2024-04-25T16:27:36.694033+0000","last_fullsized":"2024-04-25T16:27:36.694033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697810+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:36.697810+0000","last_peered":"2024-04-25T16:27:36.697810+0000","last_clean":"2024-04-25T16:27:36.697810+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:36.697810+0000","last_undegraded":"2024-04-25T16:27:36.697810+0000","last_fullsized":"2024-04-25T16:27:36.697810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764249+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:36.764249+0000","last_peered":"2024-04-25T16:27:36.764249+0000","last_clean":"2024-04-25T16:27:36.764249+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:36.764249+0000","last_undegraded":"2024-04-25T16:27:36.764249+0000","last_fullsized":"2024-04-25T16:27:36.764249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697835+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:36.697835+0000","last_peered":"2024-04-25T16:27:36.697835+0000","last_clean":"2024-04-25T16:27:36.697835+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:36.697835+0000","last_undegraded":"2024-04-25T16:27:36.697835+0000","last_fullsized":"2024-04-25T16:27:36.697835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764462+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:36.764462+0000","last_peered":"2024-04-25T16:27:36.764462+0000","last_clean":"2024-04-25T16:27:36.764462+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:36.764462+0000","last_undegraded":"2024-04-25T16:27:36.764462+0000","last_fullsized":"2024-04-25T16:27:36.764462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697522+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:36.697522+0000","last_peered":"2024-04-25T16:27:36.697522+0000","last_clean":"2024-04-25T16:27:36.697522+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:36.697522+0000","last_undegraded":"2024-04-25T16:27:36.697522+0000","last_fullsized":"2024-04-25T16:27:36.697522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697310+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:36.697310+0000","last_peered":"2024-04-25T16:27:36.697310+0000","last_clean":"2024-04-25T16:27:36.697310+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:36.697310+0000","last_undegraded":"2024-04-25T16:27:36.697310+0000","last_fullsized":"2024-04-25T16:27:36.697310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698883+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:36.698883+0000","last_peered":"2024-04-25T16:27:36.698883+0000","last_clean":"2024-04-25T16:27:36.698883+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:36.698883+0000","last_undegraded":"2024-04-25T16:27:36.698883+0000","last_fullsized":"2024-04-25T16:27:36.698883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15331,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":28232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":708870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":132453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":118046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":307200,"data_stored":130023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":117920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":118080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":892928,"data_stored":708926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":12103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:53.248 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T16:27:53.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:27:53.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.390+0000 7f1a31eb1640 1 Processor -- start 2024-04-25T16:27:53.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.390+0000 7f1a31eb1640 1 -- start start 2024-04-25T16:27:53.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.391+0000 7f1a31eb1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1a2c0b3350 0x7f1a2c08e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.391+0000 7f1a31eb1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2c0d24a0 con 0x7f1a2c0d2090 2024-04-25T16:27:53.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.391+0000 7f1a31eb1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2c0b3760 con 0x7f1a2c0d11b0 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.391+0000 7f1a31eb1640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2c0d2720 con 0x7f1a2c0b3350 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1a2c0b3350 0x7f1a2c08e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1a2c0d2090 0x7f1a2c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48008/0 (socket says 172.21.15.94:48008) 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2b7fe640 1 -- 172.21.15.94:0/4138800555 learned_addr learned my addr 172.21.15.94:0/4138800555 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2bfff640 1 -- 172.21.15.94:0/4138800555 >> v1:172.21.15.138:6789/0 conn(0x7f1a2c0d11b0 legacy=0x7f1a2c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2bfff640 1 -- 172.21.15.94:0/4138800555 >> v1:172.21.15.94:6789/0 conn(0x7f1a2c0d2090 legacy=0x7f1a2c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2bfff640 1 -- 172.21.15.94:0/4138800555 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c0d2a00 con 0x7f1a2c0b3350 2024-04-25T16:27:53.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.392+0000 7f1a2a7fc640 1 -- 172.21.15.94:0/4138800555 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448457600 0 0) 0x7f1a1c001290 con 0x7f1a2c0d2090 2024-04-25T16:27:53.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a2bfff640 1 --2- 172.21.15.94:0/4138800555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c08e080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009d90 tx=0x7f1a2402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8f8b6d2bdb2a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a2a7fc640 1 -- 172.21.15.94:0/4138800555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24031600 con 0x7f1a2c0b3350 2024-04-25T16:27:53.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a2a7fc640 1 -- 172.21.15.94:0/4138800555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a240317a0 con 0x7f1a2c0b3350 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a2a7fc640 1 -- 172.21.15.94:0/4138800555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24031b70 con 0x7f1a2c0b3350 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 -- 172.21.15.94:0/4138800555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 msgr2=0x7f1a2c08e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/4138800555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c08e080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009d90 tx=0x7f1a2402a610 comp rx=0 tx=0).stop 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 -- 172.21.15.94:0/4138800555 shutdown_connections 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/4138800555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c08e080 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 -- 172.21.15.94:0/4138800555 >> 172.21.15.94:0/4138800555 conn(0x7f1a2c0d4090 msgr2=0x7f1a2c0d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 -- 172.21.15.94:0/4138800555 shutdown_connections 2024-04-25T16:27:53.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.393+0000 7f1a31eb1640 1 -- 172.21.15.94:0/4138800555 wait complete. 2024-04-25T16:27:53.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.395+0000 7f1a31eb1640 1 Processor -- start 2024-04-25T16:27:53.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.395+0000 7f1a31eb1640 1 -- start start 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a2c0d2090 0x7f1a2c0b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a2c0d14e0 0x7f1a2c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0ced50 con 0x7f1a2c0b3350 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0d2fc0 con 0x7f1a2c0d2090 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2c0b4470 con 0x7f1a2c0d14e0 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c0d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42520/0 (socket says 172.21.15.94:42520) 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 -- 172.21.15.94:0/3280662473 learned_addr learned my addr 172.21.15.94:0/3280662473 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2bfff640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a2c0d14e0 0x7f1a2c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2affd640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a2c0d2090 0x7f1a2c0b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 -- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a2c0d14e0 msgr2=0x7f1a2c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a2c0d14e0 0x7f1a2c0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 -- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a2c0d2090 msgr2=0x7f1a2c0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a2c0d2090 0x7f1a2c0b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 -- 172.21.15.94:0/3280662473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c0cf460 con 0x7f1a2c0b3350 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2bfff640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a2c0d14e0 0x7f1a2c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2affd640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a2c0d2090 0x7f1a2c0b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a2b7fe640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c0d0fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c430 tx=0x7f1a1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c017070 con 0x7f1a2c0b3350 2024-04-25T16:27:53.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c0074e0 con 0x7f1a2c0b3350 2024-04-25T16:27:53.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c0cf740 con 0x7f1a2c0b3350 2024-04-25T16:27:53.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0059e0 con 0x7f1a2c0b3350 2024-04-25T16:27:53.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.397+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c0cfd00 con 0x7f1a2c0b3350 2024-04-25T16:27:53.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.399+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c06a180 con 0x7f1a2c0b3350 2024-04-25T16:27:53.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.399+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a1c01a070 con 0x7f1a2c0b3350 2024-04-25T16:27:53.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.399+0000 7f1a13fff640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1a180744e0 0x7f1a180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.399+0000 7f1a2affd640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1a180744e0 0x7f1a180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.399+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1a1c094a50 con 0x7f1a2c0b3350 2024-04-25T16:27:53.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.400+0000 7f1a2affd640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1a180744e0 0x7f1a180769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0b3140 tx=0x7f1a20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.404+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a2c0d24a0 con 0x7f1a2c0b3350 2024-04-25T16:27:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.580+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a2c0cdf70 con 0x7f1a180744e0 2024-04-25T16:27:53.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.588+0000 7f1a13fff640 1 -- 172.21.15.94:0/3280662473 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f1a2c0cdf70 con 0x7f1a180744e0 2024-04-25T16:27:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:53.594 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1a180744e0 msgr2=0x7f1a180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1a180744e0 0x7f1a180769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0b3140 tx=0x7f1a20008040 comp rx=0 tx=0).stop 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 msgr2=0x7f1a2c0d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c0d0fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c430 tx=0x7f1a1c00c8f0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 shutdown_connections 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f1a180744e0 0x7f1a180769f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a2c0d14e0 0x7f1a2c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a2c0d2090 0x7f1a2c0b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 --2- 172.21.15.94:0/3280662473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c0b3350 0x7f1a2c0d0fa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 >> 172.21.15.94:0/3280662473 conn(0x7f1a2c0d4090 msgr2=0x7f1a2c0b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 shutdown_connections 2024-04-25T16:27:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.592+0000 7f1a31eb1640 1 -- 172.21.15.94:0/3280662473 wait complete. 2024-04-25T16:27:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-25T16:27:53.310252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605699,"num_objects":25,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":166,"num_write_kb":1665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236280,"kb_used_data":3488,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738440,"statfs":{"total":773094113280,"available":772852162560,"internally_reserved":0,"allocated":3571712,"data_stored":2154899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10179,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002422"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699051+0000","last_change":"2024-04-25T16:26:55.557918+0000","last_active":"2024-04-25T16:27:36.699051+0000","last_peered":"2024-04-25T16:27:36.699051+0000","last_clean":"2024-04-25T16:27:36.699051+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:27:36.699051+0000","last_undegraded":"2024-04-25T16:27:36.699051+0000","last_fullsized":"2024-04-25T16:27:36.699051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694756+0000","last_change":"2024-04-25T16:26:56.563221+0000","last_active":"2024-04-25T16:27:36.694756+0000","last_peered":"2024-04-25T16:27:36.694756+0000","last_clean":"2024-04-25T16:27:36.694756+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:27:36.694756+0000","last_undegraded":"2024-04-25T16:27:36.694756+0000","last_fullsized":"2024-04-25T16:27:36.694756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693795+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:36.693795+0000","last_peered":"2024-04-25T16:27:36.693795+0000","last_clean":"2024-04-25T16:27:36.693795+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:36.693795+0000","last_undegraded":"2024-04-25T16:27:36.693795+0000","last_fullsized":"2024-04-25T16:27:36.693795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:53.108880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694849+0000","last_change":"2024-04-25T16:26:55.556890+0000","last_active":"2024-04-25T16:27:36.694849+0000","last_peered":"2024-04-25T16:27:36.694849+0000","last_clean":"2024-04-25T16:27:36.694849+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:27:36.694849+0000","last_undegraded":"2024-04-25T16:27:36.694849+0000","last_fullsized":"2024-04-25T16:27:36.694849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697915+0000","last_change":"2024-04-25T16:26:56.557991+0000","last_active":"2024-04-25T16:27:36.697915+0000","last_peered":"2024-04-25T16:27:36.697915+0000","last_clean":"2024-04-25T16:27:36.697915+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:36.697915+0000","last_undegraded":"2024-04-25T16:27:36.697915+0000","last_fullsized":"2024-04-25T16:27:36.697915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750192+0000","last_change":"2024-04-25T16:27:00.589344+0000","last_active":"2024-04-25T16:27:36.750192+0000","last_peered":"2024-04-25T16:27:36.750192+0000","last_clean":"2024-04-25T16:27:36.750192+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:27:36.750192+0000","last_undegraded":"2024-04-25T16:27:36.750192+0000","last_fullsized":"2024-04-25T16:27:36.750192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699035+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:36.699035+0000","last_peered":"2024-04-25T16:27:36.699035+0000","last_clean":"2024-04-25T16:27:36.699035+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:36.699035+0000","last_undegraded":"2024-04-25T16:27:36.699035+0000","last_fullsized":"2024-04-25T16:27:36.699035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764530+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:36.764530+0000","last_peered":"2024-04-25T16:27:36.764530+0000","last_clean":"2024-04-25T16:27:36.764530+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:36.764530+0000","last_undegraded":"2024-04-25T16:27:36.764530+0000","last_fullsized":"2024-04-25T16:27:36.764530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750366+0000","last_change":"2024-04-25T16:27:00.586673+0000","last_active":"2024-04-25T16:27:36.750366+0000","last_peered":"2024-04-25T16:27:36.750366+0000","last_clean":"2024-04-25T16:27:36.750366+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:27:36.750366+0000","last_undegraded":"2024-04-25T16:27:36.750366+0000","last_fullsized":"2024-04-25T16:27:36.750366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.260317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697999+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:36.697999+0000","last_peered":"2024-04-25T16:27:36.697999+0000","last_clean":"2024-04-25T16:27:36.697999+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:36.697999+0000","last_undegraded":"2024-04-25T16:27:36.697999+0000","last_fullsized":"2024-04-25T16:27:36.697999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764669+0000","last_change":"2024-04-25T16:26:56.565019+0000","last_active":"2024-04-25T16:27:36.764669+0000","last_peered":"2024-04-25T16:27:36.764669+0000","last_clean":"2024-04-25T16:27:36.764669+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:27:36.764669+0000","last_undegraded":"2024-04-25T16:27:36.764669+0000","last_fullsized":"2024-04-25T16:27:36.764669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750279+0000","last_change":"2024-04-25T16:27:00.588701+0000","last_active":"2024-04-25T16:27:36.750279+0000","last_peered":"2024-04-25T16:27:36.750279+0000","last_clean":"2024-04-25T16:27:36.750279+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:27:36.750279+0000","last_undegraded":"2024-04-25T16:27:36.750279+0000","last_fullsized":"2024-04-25T16:27:36.750279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.028871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697286+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:36.697286+0000","last_peered":"2024-04-25T16:27:36.697286+0000","last_clean":"2024-04-25T16:27:36.697286+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:36.697286+0000","last_undegraded":"2024-04-25T16:27:36.697286+0000","last_fullsized":"2024-04-25T16:27:36.697286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698555+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:36.698555+0000","last_peered":"2024-04-25T16:27:36.698555+0000","last_clean":"2024-04-25T16:27:36.698555+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:36.698555+0000","last_undegraded":"2024-04-25T16:27:36.698555+0000","last_fullsized":"2024-04-25T16:27:36.698555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750951+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:36.750951+0000","last_peered":"2024-04-25T16:27:36.750951+0000","last_clean":"2024-04-25T16:27:36.750951+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:36.750951+0000","last_undegraded":"2024-04-25T16:27:36.750951+0000","last_fullsized":"2024-04-25T16:27:36.750951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694265+0000","last_change":"2024-04-25T16:26:56.553462+0000","last_active":"2024-04-25T16:27:36.694265+0000","last_peered":"2024-04-25T16:27:36.694265+0000","last_clean":"2024-04-25T16:27:36.694265+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:27:36.694265+0000","last_undegraded":"2024-04-25T16:27:36.694265+0000","last_fullsized":"2024-04-25T16:27:36.694265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697095+0000","last_change":"2024-04-25T16:26:55.551818+0000","last_active":"2024-04-25T16:27:36.697095+0000","last_peered":"2024-04-25T16:27:36.697095+0000","last_clean":"2024-04-25T16:27:36.697095+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:27:36.697095+0000","last_undegraded":"2024-04-25T16:27:36.697095+0000","last_fullsized":"2024-04-25T16:27:36.697095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750759+0000","last_change":"2024-04-25T16:27:00.589584+0000","last_active":"2024-04-25T16:27:36.750759+0000","last_peered":"2024-04-25T16:27:36.750759+0000","last_clean":"2024-04-25T16:27:36.750759+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:27:36.750759+0000","last_undegraded":"2024-04-25T16:27:36.750759+0000","last_fullsized":"2024-04-25T16:27:36.750759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:37.881892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694142+0000","last_change":"2024-04-25T16:26:56.550637+0000","last_active":"2024-04-25T16:27:36.694142+0000","last_peered":"2024-04-25T16:27:36.694142+0000","last_clean":"2024-04-25T16:27:36.694142+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:27:36.694142+0000","last_undegraded":"2024-04-25T16:27:36.694142+0000","last_fullsized":"2024-04-25T16:27:36.694142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696965+0000","last_change":"2024-04-25T16:26:55.561111+0000","last_active":"2024-04-25T16:27:36.696965+0000","last_peered":"2024-04-25T16:27:36.696965+0000","last_clean":"2024-04-25T16:27:36.696965+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:27:36.696965+0000","last_undegraded":"2024-04-25T16:27:36.696965+0000","last_fullsized":"2024-04-25T16:27:36.696965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.752445+0000","last_change":"2024-04-25T16:27:36.751906+0000","last_active":"2024-04-25T16:27:36.752445+0000","last_peered":"2024-04-25T16:27:36.752445+0000","last_clean":"2024-04-25T16:27:36.752445+0000","last_became_active":"2024-04-25T16:27:36.751463+0000","last_became_peered":"2024-04-25T16:27:36.751463+0000","last_unstale":"2024-04-25T16:27:36.752445+0000","last_undegraded":"2024-04-25T16:27:36.752445+0000","last_fullsized":"2024-04-25T16:27:36.752445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750568+0000","last_change":"2024-04-25T16:26:56.550667+0000","last_active":"2024-04-25T16:27:36.750568+0000","last_peered":"2024-04-25T16:27:36.750568+0000","last_clean":"2024-04-25T16:27:36.750568+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:27:36.750568+0000","last_undegraded":"2024-04-25T16:27:36.750568+0000","last_fullsized":"2024-04-25T16:27:36.750568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:18.807392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697515+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:36.697515+0000","last_peered":"2024-04-25T16:27:36.697515+0000","last_clean":"2024-04-25T16:27:36.697515+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:36.697515+0000","last_undegraded":"2024-04-25T16:27:36.697515+0000","last_fullsized":"2024-04-25T16:27:36.697515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750847+0000","last_change":"2024-04-25T16:27:00.586312+0000","last_active":"2024-04-25T16:27:36.750847+0000","last_peered":"2024-04-25T16:27:36.750847+0000","last_clean":"2024-04-25T16:27:36.750847+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:27:36.750847+0000","last_undegraded":"2024-04-25T16:27:36.750847+0000","last_fullsized":"2024-04-25T16:27:36.750847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:11.097698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698515+0000","last_change":"2024-04-25T16:26:56.551524+0000","last_active":"2024-04-25T16:27:36.698515+0000","last_peered":"2024-04-25T16:27:36.698515+0000","last_clean":"2024-04-25T16:27:36.698515+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:27:36.698515+0000","last_undegraded":"2024-04-25T16:27:36.698515+0000","last_fullsized":"2024-04-25T16:27:36.698515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.700039+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:36.700039+0000","last_peered":"2024-04-25T16:27:36.700039+0000","last_clean":"2024-04-25T16:27:36.700039+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:36.700039+0000","last_undegraded":"2024-04-25T16:27:36.700039+0000","last_fullsized":"2024-04-25T16:27:36.700039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750332+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:36.750332+0000","last_peered":"2024-04-25T16:27:36.750332+0000","last_clean":"2024-04-25T16:27:36.750332+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:36.750332+0000","last_undegraded":"2024-04-25T16:27:36.750332+0000","last_fullsized":"2024-04-25T16:27:36.750332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:49.394849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.695158+0000","last_change":"2024-04-25T16:26:56.569109+0000","last_active":"2024-04-25T16:27:36.695158+0000","last_peered":"2024-04-25T16:27:36.695158+0000","last_clean":"2024-04-25T16:27:36.695158+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:27:36.695158+0000","last_undegraded":"2024-04-25T16:27:36.695158+0000","last_fullsized":"2024-04-25T16:27:36.695158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:41.552216+0000","last_change":"2024-04-25T16:26:55.554120+0000","last_active":"2024-04-25T16:27:41.552216+0000","last_peered":"2024-04-25T16:27:41.552216+0000","last_clean":"2024-04-25T16:27:41.552216+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:27:41.552216+0000","last_undegraded":"2024-04-25T16:27:41.552216+0000","last_fullsized":"2024-04-25T16:27:41.552216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693728+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:36.693728+0000","last_peered":"2024-04-25T16:27:36.693728+0000","last_clean":"2024-04-25T16:27:36.693728+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:36.693728+0000","last_undegraded":"2024-04-25T16:27:36.693728+0000","last_fullsized":"2024-04-25T16:27:36.693728+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697033+0000","last_change":"2024-04-25T16:26:56.545754+0000","last_active":"2024-04-25T16:27:36.697033+0000","last_peered":"2024-04-25T16:27:36.697033+0000","last_clean":"2024-04-25T16:27:36.697033+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:27:36.697033+0000","last_undegraded":"2024-04-25T16:27:36.697033+0000","last_fullsized":"2024-04-25T16:27:36.697033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698561+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:36.698561+0000","last_peered":"2024-04-25T16:27:36.698561+0000","last_clean":"2024-04-25T16:27:36.698561+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:36.698561+0000","last_undegraded":"2024-04-25T16:27:36.698561+0000","last_fullsized":"2024-04-25T16:27:36.698561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698300+0000","last_change":"2024-04-25T16:27:00.602354+0000","last_active":"2024-04-25T16:27:36.698300+0000","last_peered":"2024-04-25T16:27:36.698300+0000","last_clean":"2024-04-25T16:27:36.698300+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:27:36.698300+0000","last_undegraded":"2024-04-25T16:27:36.698300+0000","last_fullsized":"2024-04-25T16:27:36.698300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.639210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764769+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:36.764769+0000","last_peered":"2024-04-25T16:27:36.764769+0000","last_clean":"2024-04-25T16:27:36.764769+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:36.764769+0000","last_undegraded":"2024-04-25T16:27:36.764769+0000","last_fullsized":"2024-04-25T16:27:36.764769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698328+0000","last_change":"2024-04-25T16:26:55.553443+0000","last_active":"2024-04-25T16:27:36.698328+0000","last_peered":"2024-04-25T16:27:36.698328+0000","last_clean":"2024-04-25T16:27:36.698328+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:27:36.698328+0000","last_undegraded":"2024-04-25T16:27:36.698328+0000","last_fullsized":"2024-04-25T16:27:36.698328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696738+0000","last_change":"2024-04-25T16:27:00.591558+0000","last_active":"2024-04-25T16:27:36.696738+0000","last_peered":"2024-04-25T16:27:36.696738+0000","last_clean":"2024-04-25T16:27:36.696738+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:27:36.696738+0000","last_undegraded":"2024-04-25T16:27:36.696738+0000","last_fullsized":"2024-04-25T16:27:36.696738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750724+0000","last_change":"2024-04-25T16:26:56.553596+0000","last_active":"2024-04-25T16:27:36.750724+0000","last_peered":"2024-04-25T16:27:36.750724+0000","last_clean":"2024-04-25T16:27:36.750724+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:27:36.750724+0000","last_undegraded":"2024-04-25T16:27:36.750724+0000","last_fullsized":"2024-04-25T16:27:36.750724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764844+0000","last_change":"2024-04-25T16:26:55.554021+0000","last_active":"2024-04-25T16:27:36.764844+0000","last_peered":"2024-04-25T16:27:36.764844+0000","last_clean":"2024-04-25T16:27:36.764844+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:27:36.764844+0000","last_undegraded":"2024-04-25T16:27:36.764844+0000","last_fullsized":"2024-04-25T16:27:36.764844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696930+0000","last_change":"2024-04-25T16:27:00.588849+0000","last_active":"2024-04-25T16:27:36.696930+0000","last_peered":"2024-04-25T16:27:36.696930+0000","last_clean":"2024-04-25T16:27:36.696930+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:27:36.696930+0000","last_undegraded":"2024-04-25T16:27:36.696930+0000","last_fullsized":"2024-04-25T16:27:36.696930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698136+0000","last_change":"2024-04-25T16:26:56.543393+0000","last_active":"2024-04-25T16:27:36.698136+0000","last_peered":"2024-04-25T16:27:36.698136+0000","last_clean":"2024-04-25T16:27:36.698136+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:27:36.698136+0000","last_undegraded":"2024-04-25T16:27:36.698136+0000","last_fullsized":"2024-04-25T16:27:36.698136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697778+0000","last_change":"2024-04-25T16:26:55.552165+0000","last_active":"2024-04-25T16:27:36.697778+0000","last_peered":"2024-04-25T16:27:36.697778+0000","last_clean":"2024-04-25T16:27:36.697778+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:27:36.697778+0000","last_undegraded":"2024-04-25T16:27:36.697778+0000","last_fullsized":"2024-04-25T16:27:36.697778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697213+0000","last_change":"2024-04-25T16:27:00.589867+0000","last_active":"2024-04-25T16:27:36.697213+0000","last_peered":"2024-04-25T16:27:36.697213+0000","last_clean":"2024-04-25T16:27:36.697213+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:27:36.697213+0000","last_undegraded":"2024-04-25T16:27:36.697213+0000","last_fullsized":"2024-04-25T16:27:36.697213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697266+0000","last_change":"2024-04-25T16:26:56.556248+0000","last_active":"2024-04-25T16:27:36.697266+0000","last_peered":"2024-04-25T16:27:36.697266+0000","last_clean":"2024-04-25T16:27:36.697266+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:27:36.697266+0000","last_undegraded":"2024-04-25T16:27:36.697266+0000","last_fullsized":"2024-04-25T16:27:36.697266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:51.382734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697357+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:36.697357+0000","last_peered":"2024-04-25T16:27:36.697357+0000","last_clean":"2024-04-25T16:27:36.697357+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:36.697357+0000","last_undegraded":"2024-04-25T16:27:36.697357+0000","last_fullsized":"2024-04-25T16:27:36.697357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698254+0000","last_change":"2024-04-25T16:27:00.602364+0000","last_active":"2024-04-25T16:27:36.698254+0000","last_peered":"2024-04-25T16:27:36.698254+0000","last_clean":"2024-04-25T16:27:36.698254+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:27:36.698254+0000","last_undegraded":"2024-04-25T16:27:36.698254+0000","last_fullsized":"2024-04-25T16:27:36.698254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:03.528824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694153+0000","last_change":"2024-04-25T16:26:56.564700+0000","last_active":"2024-04-25T16:27:36.694153+0000","last_peered":"2024-04-25T16:27:36.694153+0000","last_clean":"2024-04-25T16:27:36.694153+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:27:36.694153+0000","last_undegraded":"2024-04-25T16:27:36.694153+0000","last_fullsized":"2024-04-25T16:27:36.694153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764525+0000","last_change":"2024-04-25T16:26:55.553968+0000","last_active":"2024-04-25T16:27:36.764525+0000","last_peered":"2024-04-25T16:27:36.764525+0000","last_clean":"2024-04-25T16:27:36.764525+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:27:36.764525+0000","last_undegraded":"2024-04-25T16:27:36.764525+0000","last_fullsized":"2024-04-25T16:27:36.764525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697495+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:36.697495+0000","last_peered":"2024-04-25T16:27:36.697495+0000","last_clean":"2024-04-25T16:27:36.697495+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:36.697495+0000","last_undegraded":"2024-04-25T16:27:36.697495+0000","last_fullsized":"2024-04-25T16:27:36.697495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698810+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:36.698810+0000","last_peered":"2024-04-25T16:27:36.698810+0000","last_clean":"2024-04-25T16:27:36.698810+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:36.698810+0000","last_undegraded":"2024-04-25T16:27:36.698810+0000","last_fullsized":"2024-04-25T16:27:36.698810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:11.509207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698193+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:36.698193+0000","last_peered":"2024-04-25T16:27:36.698193+0000","last_clean":"2024-04-25T16:27:36.698193+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:36.698193+0000","last_undegraded":"2024-04-25T16:27:36.698193+0000","last_fullsized":"2024-04-25T16:27:36.698193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697778+0000","last_change":"2024-04-25T16:26:55.561858+0000","last_active":"2024-04-25T16:27:36.697778+0000","last_peered":"2024-04-25T16:27:36.697778+0000","last_clean":"2024-04-25T16:27:36.697778+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:27:36.697778+0000","last_undegraded":"2024-04-25T16:27:36.697778+0000","last_fullsized":"2024-04-25T16:27:36.697778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697238+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:36.697238+0000","last_peered":"2024-04-25T16:27:36.697238+0000","last_clean":"2024-04-25T16:27:36.697238+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:36.697238+0000","last_undegraded":"2024-04-25T16:27:36.697238+0000","last_fullsized":"2024-04-25T16:27:36.697238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:36.822161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764579+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:36.764579+0000","last_peered":"2024-04-25T16:27:36.764579+0000","last_clean":"2024-04-25T16:27:36.764579+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:36.764579+0000","last_undegraded":"2024-04-25T16:27:36.764579+0000","last_fullsized":"2024-04-25T16:27:36.764579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:54.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697603+0000","last_change":"2024-04-25T16:26:55.551868+0000","last_active":"2024-04-25T16:27:36.697603+0000","last_peered":"2024-04-25T16:27:36.697603+0000","last_clean":"2024-04-25T16:27:36.697603+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:27:36.697603+0000","last_undegraded":"2024-04-25T16:27:36.697603+0000","last_fullsized":"2024-04-25T16:27:36.697603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.800774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698509+0000","last_change":"2024-04-25T16:27:00.586463+0000","last_active":"2024-04-25T16:27:36.698509+0000","last_peered":"2024-04-25T16:27:36.698509+0000","last_clean":"2024-04-25T16:27:36.698509+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:27:36.698509+0000","last_undegraded":"2024-04-25T16:27:36.698509+0000","last_fullsized":"2024-04-25T16:27:36.698509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:10.987478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699318+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:36.699318+0000","last_peered":"2024-04-25T16:27:36.699318+0000","last_clean":"2024-04-25T16:27:36.699318+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:36.699318+0000","last_undegraded":"2024-04-25T16:27:36.699318+0000","last_fullsized":"2024-04-25T16:27:36.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697295+0000","last_change":"2024-04-25T16:26:55.551363+0000","last_active":"2024-04-25T16:27:36.697295+0000","last_peered":"2024-04-25T16:27:36.697295+0000","last_clean":"2024-04-25T16:27:36.697295+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:27:36.697295+0000","last_undegraded":"2024-04-25T16:27:36.697295+0000","last_fullsized":"2024-04-25T16:27:36.697295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.763861+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:36.763861+0000","last_peered":"2024-04-25T16:27:36.763861+0000","last_clean":"2024-04-25T16:27:36.763861+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:36.763861+0000","last_undegraded":"2024-04-25T16:27:36.763861+0000","last_fullsized":"2024-04-25T16:27:36.763861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698846+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:36.698846+0000","last_peered":"2024-04-25T16:27:36.698846+0000","last_clean":"2024-04-25T16:27:36.698846+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:36.698846+0000","last_undegraded":"2024-04-25T16:27:36.698846+0000","last_fullsized":"2024-04-25T16:27:36.698846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751170+0000","last_change":"2024-04-25T16:26:55.553075+0000","last_active":"2024-04-25T16:27:36.751170+0000","last_peered":"2024-04-25T16:27:36.751170+0000","last_clean":"2024-04-25T16:27:36.751170+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:27:36.751170+0000","last_undegraded":"2024-04-25T16:27:36.751170+0000","last_fullsized":"2024-04-25T16:27:36.751170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750149+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:36.750149+0000","last_peered":"2024-04-25T16:27:36.750149+0000","last_clean":"2024-04-25T16:27:36.750149+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:36.750149+0000","last_undegraded":"2024-04-25T16:27:36.750149+0000","last_fullsized":"2024-04-25T16:27:36.750149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:43.417673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698175+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:36.698175+0000","last_peered":"2024-04-25T16:27:36.698175+0000","last_clean":"2024-04-25T16:27:36.698175+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:36.698175+0000","last_undegraded":"2024-04-25T16:27:36.698175+0000","last_fullsized":"2024-04-25T16:27:36.698175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697542+0000","last_change":"2024-04-25T16:26:55.552088+0000","last_active":"2024-04-25T16:27:36.697542+0000","last_peered":"2024-04-25T16:27:36.697542+0000","last_clean":"2024-04-25T16:27:36.697542+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:27:36.697542+0000","last_undegraded":"2024-04-25T16:27:36.697542+0000","last_fullsized":"2024-04-25T16:27:36.697542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:44.340830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698129+0000","last_change":"2024-04-25T16:27:00.587838+0000","last_active":"2024-04-25T16:27:36.698129+0000","last_peered":"2024-04-25T16:27:36.698129+0000","last_clean":"2024-04-25T16:27:36.698129+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:27:36.698129+0000","last_undegraded":"2024-04-25T16:27:36.698129+0000","last_fullsized":"2024-04-25T16:27:36.698129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697530+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:36.697530+0000","last_peered":"2024-04-25T16:27:36.697530+0000","last_clean":"2024-04-25T16:27:36.697530+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:36.697530+0000","last_undegraded":"2024-04-25T16:27:36.697530+0000","last_fullsized":"2024-04-25T16:27:36.697530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697833+0000","last_change":"2024-04-25T16:26:55.554459+0000","last_active":"2024-04-25T16:27:36.697833+0000","last_peered":"2024-04-25T16:27:36.697833+0000","last_clean":"2024-04-25T16:27:36.697833+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:27:36.697833+0000","last_undegraded":"2024-04-25T16:27:36.697833+0000","last_fullsized":"2024-04-25T16:27:36.697833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694937+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:36.694937+0000","last_peered":"2024-04-25T16:27:36.694937+0000","last_clean":"2024-04-25T16:27:36.694937+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:36.694937+0000","last_undegraded":"2024-04-25T16:27:36.694937+0000","last_fullsized":"2024-04-25T16:27:36.694937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.396901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698423+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:36.698423+0000","last_peered":"2024-04-25T16:27:36.698423+0000","last_clean":"2024-04-25T16:27:36.698423+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:36.698423+0000","last_undegraded":"2024-04-25T16:27:36.698423+0000","last_fullsized":"2024-04-25T16:27:36.698423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764677+0000","last_change":"2024-04-25T16:26:55.558982+0000","last_active":"2024-04-25T16:27:36.764677+0000","last_peered":"2024-04-25T16:27:36.764677+0000","last_clean":"2024-04-25T16:27:36.764677+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:27:36.764677+0000","last_undegraded":"2024-04-25T16:27:36.764677+0000","last_fullsized":"2024-04-25T16:27:36.764677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:44.103837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694274+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:36.694274+0000","last_peered":"2024-04-25T16:27:36.694274+0000","last_clean":"2024-04-25T16:27:36.694274+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:36.694274+0000","last_undegraded":"2024-04-25T16:27:36.694274+0000","last_fullsized":"2024-04-25T16:27:36.694274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:33.786813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698335+0000","last_change":"2024-04-25T16:26:56.558158+0000","last_active":"2024-04-25T16:27:36.698335+0000","last_peered":"2024-04-25T16:27:36.698335+0000","last_clean":"2024-04-25T16:27:36.698335+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:27:36.698335+0000","last_undegraded":"2024-04-25T16:27:36.698335+0000","last_fullsized":"2024-04-25T16:27:36.698335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698767+0000","last_change":"2024-04-25T16:26:55.547657+0000","last_active":"2024-04-25T16:27:36.698767+0000","last_peered":"2024-04-25T16:27:36.698767+0000","last_clean":"2024-04-25T16:27:36.698767+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:27:36.698767+0000","last_undegraded":"2024-04-25T16:27:36.698767+0000","last_fullsized":"2024-04-25T16:27:36.698767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:41.163897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764506+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:36.764506+0000","last_peered":"2024-04-25T16:27:36.764506+0000","last_clean":"2024-04-25T16:27:36.764506+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:36.764506+0000","last_undegraded":"2024-04-25T16:27:36.764506+0000","last_fullsized":"2024-04-25T16:27:36.764506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698073+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:36.698073+0000","last_peered":"2024-04-25T16:27:36.698073+0000","last_clean":"2024-04-25T16:27:36.698073+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:36.698073+0000","last_undegraded":"2024-04-25T16:27:36.698073+0000","last_fullsized":"2024-04-25T16:27:36.698073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'21","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:47.513651+0000","last_change":"2024-04-25T16:26:55.552906+0000","last_active":"2024-04-25T16:27:47.513651+0000","last_peered":"2024-04-25T16:27:47.513651+0000","last_clean":"2024-04-25T16:27:47.513651+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:27:47.513651+0000","last_undegraded":"2024-04-25T16:27:47.513651+0000","last_fullsized":"2024-04-25T16:27:47.513651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696855+0000","last_change":"2024-04-25T16:27:00.584326+0000","last_active":"2024-04-25T16:27:36.696855+0000","last_peered":"2024-04-25T16:27:36.696855+0000","last_clean":"2024-04-25T16:27:36.696855+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:27:36.696855+0000","last_undegraded":"2024-04-25T16:27:36.696855+0000","last_fullsized":"2024-04-25T16:27:36.696855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.149344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.701265+0000","last_change":"2024-04-25T16:27:36.700437+0000","last_active":"2024-04-25T16:27:36.701265+0000","last_peered":"2024-04-25T16:27:36.701265+0000","last_clean":"2024-04-25T16:27:36.701265+0000","last_became_active":"2024-04-25T16:27:36.700105+0000","last_became_peered":"2024-04-25T16:27:36.700105+0000","last_unstale":"2024-04-25T16:27:36.701265+0000","last_undegraded":"2024-04-25T16:27:36.701265+0000","last_fullsized":"2024-04-25T16:27:36.701265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697453+0000","last_change":"2024-04-25T16:26:55.559245+0000","last_active":"2024-04-25T16:27:36.697453+0000","last_peered":"2024-04-25T16:27:36.697453+0000","last_clean":"2024-04-25T16:27:36.697453+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:27:36.697453+0000","last_undegraded":"2024-04-25T16:27:36.697453+0000","last_fullsized":"2024-04-25T16:27:36.697453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:17.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750587+0000","last_change":"2024-04-25T16:27:00.588944+0000","last_active":"2024-04-25T16:27:36.750587+0000","last_peered":"2024-04-25T16:27:36.750587+0000","last_clean":"2024-04-25T16:27:36.750587+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:27:36.750587+0000","last_undegraded":"2024-04-25T16:27:36.750587+0000","last_fullsized":"2024-04-25T16:27:36.750587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:04.899160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698936+0000","last_change":"2024-04-25T16:26:56.558303+0000","last_active":"2024-04-25T16:27:36.698936+0000","last_peered":"2024-04-25T16:27:36.698936+0000","last_clean":"2024-04-25T16:27:36.698936+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:27:36.698936+0000","last_undegraded":"2024-04-25T16:27:36.698936+0000","last_fullsized":"2024-04-25T16:27:36.698936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764563+0000","last_change":"2024-04-25T16:26:55.558912+0000","last_active":"2024-04-25T16:27:36.764563+0000","last_peered":"2024-04-25T16:27:36.764563+0000","last_clean":"2024-04-25T16:27:36.764563+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:27:36.764563+0000","last_undegraded":"2024-04-25T16:27:36.764563+0000","last_fullsized":"2024-04-25T16:27:36.764563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:41.196000+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:41.196000+0000","last_peered":"2024-04-25T16:27:41.196000+0000","last_clean":"2024-04-25T16:27:41.196000+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:41.196000+0000","last_undegraded":"2024-04-25T16:27:41.196000+0000","last_fullsized":"2024-04-25T16:27:41.196000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:34.264471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764463+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:36.764463+0000","last_peered":"2024-04-25T16:27:36.764463+0000","last_clean":"2024-04-25T16:27:36.764463+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:36.764463+0000","last_undegraded":"2024-04-25T16:27:36.764463+0000","last_fullsized":"2024-04-25T16:27:36.764463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699948+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:36.699948+0000","last_peered":"2024-04-25T16:27:36.699948+0000","last_clean":"2024-04-25T16:27:36.699948+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:36.699948+0000","last_undegraded":"2024-04-25T16:27:36.699948+0000","last_fullsized":"2024-04-25T16:27:36.699948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693864+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:36.693864+0000","last_peered":"2024-04-25T16:27:36.693864+0000","last_clean":"2024-04-25T16:27:36.693864+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:36.693864+0000","last_undegraded":"2024-04-25T16:27:36.693864+0000","last_fullsized":"2024-04-25T16:27:36.693864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:49.194767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.702563+0000","last_change":"2024-04-25T16:27:36.702563+0000","last_active":"2024-04-25T16:27:36.702563+0000","last_peered":"2024-04-25T16:27:36.702563+0000","last_clean":"2024-04-25T16:27:36.702563+0000","last_became_active":"2024-04-25T16:27:36.702077+0000","last_became_peered":"2024-04-25T16:27:36.702077+0000","last_unstale":"2024-04-25T16:27:36.702563+0000","last_undegraded":"2024-04-25T16:27:36.702563+0000","last_fullsized":"2024-04-25T16:27:36.702563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:48.724249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,2],"acting":[0,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.701750+0000","last_change":"2024-04-25T16:27:36.701750+0000","last_active":"2024-04-25T16:27:36.701750+0000","last_peered":"2024-04-25T16:27:36.701750+0000","last_clean":"2024-04-25T16:27:36.701750+0000","last_became_active":"2024-04-25T16:27:36.701427+0000","last_became_peered":"2024-04-25T16:27:36.701427+0000","last_unstale":"2024-04-25T16:27:36.701750+0000","last_undegraded":"2024-04-25T16:27:36.701750+0000","last_fullsized":"2024-04-25T16:27:36.701750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:44.695661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764137+0000","last_change":"2024-04-25T16:26:55.556301+0000","last_active":"2024-04-25T16:27:36.764137+0000","last_peered":"2024-04-25T16:27:36.764137+0000","last_clean":"2024-04-25T16:27:36.764137+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:27:36.764137+0000","last_undegraded":"2024-04-25T16:27:36.764137+0000","last_fullsized":"2024-04-25T16:27:36.764137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.766198+0000","last_change":"2024-04-25T16:27:36.766198+0000","last_active":"2024-04-25T16:27:36.766198+0000","last_peered":"2024-04-25T16:27:36.766198+0000","last_clean":"2024-04-25T16:27:36.766198+0000","last_became_active":"2024-04-25T16:27:36.765669+0000","last_became_peered":"2024-04-25T16:27:36.765669+0000","last_unstale":"2024-04-25T16:27:36.766198+0000","last_undegraded":"2024-04-25T16:27:36.766198+0000","last_fullsized":"2024-04-25T16:27:36.766198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:30.911634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694021+0000","last_change":"2024-04-25T16:26:56.550743+0000","last_active":"2024-04-25T16:27:36.694021+0000","last_peered":"2024-04-25T16:27:36.694021+0000","last_clean":"2024-04-25T16:27:36.694021+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:27:36.694021+0000","last_undegraded":"2024-04-25T16:27:36.694021+0000","last_fullsized":"2024-04-25T16:27:36.694021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694344+0000","last_change":"2024-04-25T16:26:55.554919+0000","last_active":"2024-04-25T16:27:36.694344+0000","last_peered":"2024-04-25T16:27:36.694344+0000","last_clean":"2024-04-25T16:27:36.694344+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:27:36.694344+0000","last_undegraded":"2024-04-25T16:27:36.694344+0000","last_fullsized":"2024-04-25T16:27:36.694344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:46.954866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698727+0000","last_change":"2024-04-25T16:26:55.551870+0000","last_active":"2024-04-25T16:27:36.698727+0000","last_peered":"2024-04-25T16:27:36.698727+0000","last_clean":"2024-04-25T16:27:36.698727+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:27:36.698727+0000","last_undegraded":"2024-04-25T16:27:36.698727+0000","last_fullsized":"2024-04-25T16:27:36.698727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698703+0000","last_change":"2024-04-25T16:26:56.556685+0000","last_active":"2024-04-25T16:27:36.698703+0000","last_peered":"2024-04-25T16:27:36.698703+0000","last_clean":"2024-04-25T16:27:36.698703+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:27:36.698703+0000","last_undegraded":"2024-04-25T16:27:36.698703+0000","last_fullsized":"2024-04-25T16:27:36.698703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697541+0000","last_change":"2024-04-25T16:27:00.588199+0000","last_active":"2024-04-25T16:27:36.697541+0000","last_peered":"2024-04-25T16:27:36.697541+0000","last_clean":"2024-04-25T16:27:36.697541+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:27:36.697541+0000","last_undegraded":"2024-04-25T16:27:36.697541+0000","last_fullsized":"2024-04-25T16:27:36.697541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.411321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694183+0000","last_change":"2024-04-25T16:26:55.559106+0000","last_active":"2024-04-25T16:27:36.694183+0000","last_peered":"2024-04-25T16:27:36.694183+0000","last_clean":"2024-04-25T16:27:36.694183+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:27:36.694183+0000","last_undegraded":"2024-04-25T16:27:36.694183+0000","last_fullsized":"2024-04-25T16:27:36.694183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:07.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750593+0000","last_change":"2024-04-25T16:26:56.553040+0000","last_active":"2024-04-25T16:27:36.750593+0000","last_peered":"2024-04-25T16:27:36.750593+0000","last_clean":"2024-04-25T16:27:36.750593+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:27:36.750593+0000","last_undegraded":"2024-04-25T16:27:36.750593+0000","last_fullsized":"2024-04-25T16:27:36.750593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:18.085936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694424+0000","last_change":"2024-04-25T16:27:00.584280+0000","last_active":"2024-04-25T16:27:36.694424+0000","last_peered":"2024-04-25T16:27:36.694424+0000","last_clean":"2024-04-25T16:27:36.694424+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:27:36.694424+0000","last_undegraded":"2024-04-25T16:27:36.694424+0000","last_fullsized":"2024-04-25T16:27:36.694424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:05.764023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697403+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:36.697403+0000","last_peered":"2024-04-25T16:27:36.697403+0000","last_clean":"2024-04-25T16:27:36.697403+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:36.697403+0000","last_undegraded":"2024-04-25T16:27:36.697403+0000","last_fullsized":"2024-04-25T16:27:36.697403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697750+0000","last_change":"2024-04-25T16:26:56.556813+0000","last_active":"2024-04-25T16:27:36.697750+0000","last_peered":"2024-04-25T16:27:36.697750+0000","last_clean":"2024-04-25T16:27:36.697750+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:27:36.697750+0000","last_undegraded":"2024-04-25T16:27:36.697750+0000","last_fullsized":"2024-04-25T16:27:36.697750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:07.857911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.702267+0000","last_change":"2024-04-25T16:27:36.701488+0000","last_active":"2024-04-25T16:27:36.702267+0000","last_peered":"2024-04-25T16:27:36.702267+0000","last_clean":"2024-04-25T16:27:36.702267+0000","last_became_active":"2024-04-25T16:27:36.701156+0000","last_became_peered":"2024-04-25T16:27:36.701156+0000","last_unstale":"2024-04-25T16:27:36.702267+0000","last_undegraded":"2024-04-25T16:27:36.702267+0000","last_fullsized":"2024-04-25T16:27:36.702267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698662+0000","last_change":"2024-04-25T16:26:56.554657+0000","last_active":"2024-04-25T16:27:36.698662+0000","last_peered":"2024-04-25T16:27:36.698662+0000","last_clean":"2024-04-25T16:27:36.698662+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:27:36.698662+0000","last_undegraded":"2024-04-25T16:27:36.698662+0000","last_fullsized":"2024-04-25T16:27:36.698662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694391+0000","last_change":"2024-04-25T16:26:56.556329+0000","last_active":"2024-04-25T16:27:36.694391+0000","last_peered":"2024-04-25T16:27:36.694391+0000","last_clean":"2024-04-25T16:27:36.694391+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:27:36.694391+0000","last_undegraded":"2024-04-25T16:27:36.694391+0000","last_fullsized":"2024-04-25T16:27:36.694391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696935+0000","last_change":"2024-04-25T16:26:56.552326+0000","last_active":"2024-04-25T16:27:36.696935+0000","last_peered":"2024-04-25T16:27:36.696935+0000","last_clean":"2024-04-25T16:27:36.696935+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:27:36.696935+0000","last_undegraded":"2024-04-25T16:27:36.696935+0000","last_fullsized":"2024-04-25T16:27:36.696935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699512+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:36.699512+0000","last_peered":"2024-04-25T16:27:36.699512+0000","last_clean":"2024-04-25T16:27:36.699512+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:36.699512+0000","last_undegraded":"2024-04-25T16:27:36.699512+0000","last_fullsized":"2024-04-25T16:27:36.699512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.695043+0000","last_change":"2024-04-25T16:26:56.553400+0000","last_active":"2024-04-25T16:27:36.695043+0000","last_peered":"2024-04-25T16:27:36.695043+0000","last_clean":"2024-04-25T16:27:36.695043+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:27:36.695043+0000","last_undegraded":"2024-04-25T16:27:36.695043+0000","last_fullsized":"2024-04-25T16:27:36.695043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:02.815120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764019+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:36.764019+0000","last_peered":"2024-04-25T16:27:36.764019+0000","last_clean":"2024-04-25T16:27:36.764019+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:36.764019+0000","last_undegraded":"2024-04-25T16:27:36.764019+0000","last_fullsized":"2024-04-25T16:27:36.764019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.000284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694199+0000","last_change":"2024-04-25T16:26:56.557842+0000","last_active":"2024-04-25T16:27:36.694199+0000","last_peered":"2024-04-25T16:27:36.694199+0000","last_clean":"2024-04-25T16:27:36.694199+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:27:36.694199+0000","last_undegraded":"2024-04-25T16:27:36.694199+0000","last_fullsized":"2024-04-25T16:27:36.694199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764266+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:36.764266+0000","last_peered":"2024-04-25T16:27:36.764266+0000","last_clean":"2024-04-25T16:27:36.764266+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:36.764266+0000","last_undegraded":"2024-04-25T16:27:36.764266+0000","last_fullsized":"2024-04-25T16:27:36.764266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698420+0000","last_change":"2024-04-25T16:26:56.562970+0000","last_active":"2024-04-25T16:27:36.698420+0000","last_peered":"2024-04-25T16:27:36.698420+0000","last_clean":"2024-04-25T16:27:36.698420+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:27:36.698420+0000","last_undegraded":"2024-04-25T16:27:36.698420+0000","last_fullsized":"2024-04-25T16:27:36.698420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698766+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:36.698766+0000","last_peered":"2024-04-25T16:27:36.698766+0000","last_clean":"2024-04-25T16:27:36.698766+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:36.698766+0000","last_undegraded":"2024-04-25T16:27:36.698766+0000","last_fullsized":"2024-04-25T16:27:36.698766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:58.111434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698354+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:36.698354+0000","last_peered":"2024-04-25T16:27:36.698354+0000","last_clean":"2024-04-25T16:27:36.698354+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:36.698354+0000","last_undegraded":"2024-04-25T16:27:36.698354+0000","last_fullsized":"2024-04-25T16:27:36.698354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697177+0000","last_change":"2024-04-25T16:26:56.555519+0000","last_active":"2024-04-25T16:27:36.697177+0000","last_peered":"2024-04-25T16:27:36.697177+0000","last_clean":"2024-04-25T16:27:36.697177+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:27:36.697177+0000","last_undegraded":"2024-04-25T16:27:36.697177+0000","last_fullsized":"2024-04-25T16:27:36.697177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.662809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764875+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:36.764875+0000","last_peered":"2024-04-25T16:27:36.764875+0000","last_clean":"2024-04-25T16:27:36.764875+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:36.764875+0000","last_undegraded":"2024-04-25T16:27:36.764875+0000","last_fullsized":"2024-04-25T16:27:36.764875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764030+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:36.764030+0000","last_peered":"2024-04-25T16:27:36.764030+0000","last_clean":"2024-04-25T16:27:36.764030+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:36.764030+0000","last_undegraded":"2024-04-25T16:27:36.764030+0000","last_fullsized":"2024-04-25T16:27:36.764030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.696865+0000","last_change":"2024-04-25T16:26:56.558767+0000","last_active":"2024-04-25T16:27:36.696865+0000","last_peered":"2024-04-25T16:27:36.696865+0000","last_clean":"2024-04-25T16:27:36.696865+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:27:36.696865+0000","last_undegraded":"2024-04-25T16:27:36.696865+0000","last_fullsized":"2024-04-25T16:27:36.696865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698226+0000","last_change":"2024-04-25T16:26:56.550591+0000","last_active":"2024-04-25T16:27:36.698226+0000","last_peered":"2024-04-25T16:27:36.698226+0000","last_clean":"2024-04-25T16:27:36.698226+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:27:36.698226+0000","last_undegraded":"2024-04-25T16:27:36.698226+0000","last_fullsized":"2024-04-25T16:27:36.698226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:20.272235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694041+0000","last_change":"2024-04-25T16:26:56.563126+0000","last_active":"2024-04-25T16:27:36.694041+0000","last_peered":"2024-04-25T16:27:36.694041+0000","last_clean":"2024-04-25T16:27:36.694041+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:27:36.694041+0000","last_undegraded":"2024-04-25T16:27:36.694041+0000","last_fullsized":"2024-04-25T16:27:36.694041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:55.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751028+0000","last_change":"2024-04-25T16:26:56.558056+0000","last_active":"2024-04-25T16:27:36.751028+0000","last_peered":"2024-04-25T16:27:36.751028+0000","last_clean":"2024-04-25T16:27:36.751028+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:27:36.751028+0000","last_undegraded":"2024-04-25T16:27:36.751028+0000","last_fullsized":"2024-04-25T16:27:36.751028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:36.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699047+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:36.699047+0000","last_peered":"2024-04-25T16:27:36.699047+0000","last_clean":"2024-04-25T16:27:36.699047+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:36.699047+0000","last_undegraded":"2024-04-25T16:27:36.699047+0000","last_fullsized":"2024-04-25T16:27:36.699047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:21.793154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699390+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:36.699390+0000","last_peered":"2024-04-25T16:27:36.699390+0000","last_clean":"2024-04-25T16:27:36.699390+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:36.699390+0000","last_undegraded":"2024-04-25T16:27:36.699390+0000","last_fullsized":"2024-04-25T16:27:36.699390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:58.657142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.703786+0000","last_change":"2024-04-25T16:27:36.703786+0000","last_active":"2024-04-25T16:27:36.703786+0000","last_peered":"2024-04-25T16:27:36.703786+0000","last_clean":"2024-04-25T16:27:36.703786+0000","last_became_active":"2024-04-25T16:27:36.703485+0000","last_became_peered":"2024-04-25T16:27:36.703485+0000","last_unstale":"2024-04-25T16:27:36.703786+0000","last_undegraded":"2024-04-25T16:27:36.703786+0000","last_fullsized":"2024-04-25T16:27:36.703786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:51.980383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694033+0000","last_change":"2024-04-25T16:26:56.564462+0000","last_active":"2024-04-25T16:27:36.694033+0000","last_peered":"2024-04-25T16:27:36.694033+0000","last_clean":"2024-04-25T16:27:36.694033+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:27:36.694033+0000","last_undegraded":"2024-04-25T16:27:36.694033+0000","last_fullsized":"2024-04-25T16:27:36.694033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.231729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697810+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:36.697810+0000","last_peered":"2024-04-25T16:27:36.697810+0000","last_clean":"2024-04-25T16:27:36.697810+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:36.697810+0000","last_undegraded":"2024-04-25T16:27:36.697810+0000","last_fullsized":"2024-04-25T16:27:36.697810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764249+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:36.764249+0000","last_peered":"2024-04-25T16:27:36.764249+0000","last_clean":"2024-04-25T16:27:36.764249+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:36.764249+0000","last_undegraded":"2024-04-25T16:27:36.764249+0000","last_fullsized":"2024-04-25T16:27:36.764249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697835+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:36.697835+0000","last_peered":"2024-04-25T16:27:36.697835+0000","last_clean":"2024-04-25T16:27:36.697835+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:36.697835+0000","last_undegraded":"2024-04-25T16:27:36.697835+0000","last_fullsized":"2024-04-25T16:27:36.697835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764462+0000","last_change":"2024-04-25T16:26:56.565416+0000","last_active":"2024-04-25T16:27:36.764462+0000","last_peered":"2024-04-25T16:27:36.764462+0000","last_clean":"2024-04-25T16:27:36.764462+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:27:36.764462+0000","last_undegraded":"2024-04-25T16:27:36.764462+0000","last_fullsized":"2024-04-25T16:27:36.764462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697522+0000","last_change":"2024-04-25T16:26:56.546547+0000","last_active":"2024-04-25T16:27:36.697522+0000","last_peered":"2024-04-25T16:27:36.697522+0000","last_clean":"2024-04-25T16:27:36.697522+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:27:36.697522+0000","last_undegraded":"2024-04-25T16:27:36.697522+0000","last_fullsized":"2024-04-25T16:27:36.697522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:22.721232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697310+0000","last_change":"2024-04-25T16:26:56.548909+0000","last_active":"2024-04-25T16:27:36.697310+0000","last_peered":"2024-04-25T16:27:36.697310+0000","last_clean":"2024-04-25T16:27:36.697310+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:27:36.697310+0000","last_undegraded":"2024-04-25T16:27:36.697310+0000","last_fullsized":"2024-04-25T16:27:36.697310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:38.145352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698883+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:36.698883+0000","last_peered":"2024-04-25T16:27:36.698883+0000","last_clean":"2024-04-25T16:27:36.698883+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:36.698883+0000","last_undegraded":"2024-04-25T16:27:36.698883+0000","last_fullsized":"2024-04-25T16:27:36.698883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15331,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":30944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":708870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":132453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":118046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":311296,"data_stored":132594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":117920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":118080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":892928,"data_stored":708926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:27:53.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:27:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.755+0000 7fa72c653640 1 Processor -- start 2024-04-25T16:27:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.755+0000 7fa72c653640 1 -- start start 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72c653640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa7240b0c60 0x7fa7240b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72c653640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7240b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72c653640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240b1f00 con 0x7fa7240b9460 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72c653640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa7240b9870 con 0x7fa7240b0c60 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72c653640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7240b6630 con 0x7fa7240b15b0 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72a3ef640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa7240b9460 0x7fa7240d1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48024/0 (socket says 172.21.15.94:48024) 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa729bee640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa7240b0c60 0x7fa7240b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72a3ef640 1 -- 172.21.15.94:0/1757745466 learned_addr learned my addr 172.21.15.94:0/1757745466 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.756+0000 7fa72abf0640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7240b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa729bee640 1 -- 172.21.15.94:0/1757745466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 msgr2=0x7fa7240b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa729bee640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7240b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa729bee640 1 -- 172.21.15.94:0/1757745466 >> v1:172.21.15.94:6789/0 conn(0x7fa7240b9460 legacy=0x7fa7240d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa729bee640 1 -- 172.21.15.94:0/1757745466 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72409d1f0 con 0x7fa7240b0c60 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa7293ed640 1 -- 172.21.15.94:0/1757745466 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447676079 0 0) 0x7fa7240b1f00 con 0x7fa7240b9460 2024-04-25T16:27:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa72abf0640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7240b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa729bee640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b0c60 0x7fa7240b1070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097a0 tx=0x7fa71402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d178e01767a306d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa7293ed640 1 -- 172.21.15.94:0/1757745466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714037070 con 0x7fa7240b0c60 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa7293ed640 1 -- 172.21.15.94:0/1757745466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714032440 con 0x7fa7240b0c60 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.757+0000 7fa7293ed640 1 -- 172.21.15.94:0/1757745466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714032810 con 0x7fa7240b0c60 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 -- 172.21.15.94:0/1757745466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b0c60 msgr2=0x7fa7240b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b0c60 0x7fa7240b1070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097a0 tx=0x7fa71402a180 comp rx=0 tx=0).stop 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 -- 172.21.15.94:0/1757745466 shutdown_connections 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7240b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 --2- 172.21.15.94:0/1757745466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b0c60 0x7fa7240b1070 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 -- 172.21.15.94:0/1757745466 >> 172.21.15.94:0/1757745466 conn(0x7fa7240b5a70 msgr2=0x7fa7240b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 -- 172.21.15.94:0/1757745466 shutdown_connections 2024-04-25T16:27:53.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.758+0000 7fa72c653640 1 -- 172.21.15.94:0/1757745466 wait complete. 2024-04-25T16:27:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.760+0000 7fa72c653640 1 Processor -- start 2024-04-25T16:27:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.760+0000 7fa72c653640 1 -- start start 2024-04-25T16:27:53.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72c653640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7240b0c60 0x7fa7241f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72c653640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7241da140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72c653640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 0x7fa7241f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72c653640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240b6910 con 0x7fa7240b9460 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72c653640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240d23b0 con 0x7fa7240b0c60 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72c653640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7240b2e60 con 0x7fa7240b15b0 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72abf0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 0x7fa7241f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72abf0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 0x7fa7241f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42528/0 (socket says 172.21.15.94:42528) 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.761+0000 7fa72abf0640 1 -- 172.21.15.94:0/4142606652 learned_addr learned my addr 172.21.15.94:0/4142606652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa729bee640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7241da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72a3ef640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7240b0c60 0x7fa7241f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72abf0640 1 -- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 msgr2=0x7fa7241da140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72abf0640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7241da140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72abf0640 1 -- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7240b0c60 msgr2=0x7fa7241f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72abf0640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7240b0c60 0x7fa7241f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72abf0640 1 -- 172.21.15.94:0/4142606652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7241da680 con 0x7fa7240b9460 2024-04-25T16:27:53.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72a3ef640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7240b0c60 0x7fa7241f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72abf0640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 0x7fa7241f4600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00d800 tx=0x7fa71c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c004030 con 0x7fa7240b9460 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71c0041d0 con 0x7fa7240b9460 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7241f5330 con 0x7fa7240b9460 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0096a0 con 0x7fa7240b9460 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.762+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7240a0840 con 0x7fa7240b9460 2024-04-25T16:27:53.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.763+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72406a180 con 0x7fa7240b9460 2024-04-25T16:27:53.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.764+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa71c009840 con 0x7fa7240b9460 2024-04-25T16:27:53.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.764+0000 7fa71affd640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:53.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.764+0000 7fa72a3ef640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:53.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.764+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa71c095940 con 0x7fa7240b9460 2024-04-25T16:27:53.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.765+0000 7fa72a3ef640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7200012e0 tx=0x7fa720005b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:53.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.769+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7241da680 con 0x7fa7240b9460 2024-04-25T16:27:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.943+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa724072510 con 0x7fa7240b9460 2024-04-25T16:27:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.944+0000 7fa71affd640 1 -- 172.21.15.94:0/4142606652 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18868 (secure 0 0 0) 0x7fa71c05fd60 con 0x7fa7240b9460 2024-04-25T16:27:53.945 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:27:53.945 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:36.681885+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 msgr2=0x7fa7080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7200012e0 tx=0x7fa720005b90 comp rx=0 tx=0).stop 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 msgr2=0x7fa7241f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 0x7fa7241f4600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00d800 tx=0x7fa71c00dcc0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 shutdown_connections 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7080744b0 0x7fa7080769c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7240b9460 0x7fa7241f4600 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7240b15b0 0x7fa7241da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 --2- 172.21.15.94:0/4142606652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7240b0c60 0x7fa7241f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:53.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 >> 172.21.15.94:0/4142606652 conn(0x7fa7240b5a70 msgr2=0x7fa7240d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:53.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 shutdown_connections 2024-04-25T16:27:53.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:53.946+0000 7fa72c653640 1 -- 172.21.15.94:0/4142606652 wait complete. 2024-04-25T16:27:54.959 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T16:27:54.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:27:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.084+0000 7fbe0a1b7640 1 Processor -- start 2024-04-25T16:27:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.084+0000 7fbe0a1b7640 1 -- start start 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe0a1b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe040d4a80 0x7fbe041642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe0a1b7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbe041608b0 0x7fbe041647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe0a1b7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe040d4f80 con 0x7fbe04164d30 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe0a1b7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbe04165140 con 0x7fbe041608b0 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe0a1b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe040d5260 con 0x7fbe040d4a80 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe02ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbe041608b0 0x7fbe041647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe03fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbe04164d30 0x7fbe040d0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48038/0 (socket says 172.21.15.94:48038) 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe02ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbe041608b0 0x7fbe041647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42540/0 (socket says 172.21.15.94:42540) 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe03fff640 1 -- 172.21.15.94:0/3068275417 learned_addr learned my addr 172.21.15.94:0/3068275417 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe037fe640 1 --2- 172.21.15.94:0/3068275417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe040d4a80 0x7fbe041642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe02ffd640 1 -- 172.21.15.94:0/3068275417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe040d4a80 msgr2=0x7fbe041642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe02ffd640 1 --2- 172.21.15.94:0/3068275417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe040d4a80 0x7fbe041642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe02ffd640 1 -- 172.21.15.94:0/3068275417 >> v1:172.21.15.94:6789/0 conn(0x7fbe04164d30 legacy=0x7fbe040d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe02ffd640 1 -- 172.21.15.94:0/3068275417 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe040d5540 con 0x7fbe041608b0 2024-04-25T16:27:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.086+0000 7fbe027fc640 1 -- 172.21.15.94:0/3068275417 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346929818 0 0) 0x7fbe040d4f80 con 0x7fbe04164d30 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.087+0000 7fbe02ffd640 1 --2- 172.21.15.94:0/3068275417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe041647f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80097a0 tx=0x7fbdf802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5327aaa76f488 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.087+0000 7fbe027fc640 1 -- 172.21.15.94:0/3068275417 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8037070 con 0x7fbe041608b0 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.087+0000 7fbe027fc640 1 -- 172.21.15.94:0/3068275417 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf8032440 con 0x7fbe041608b0 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.087+0000 7fbe027fc640 1 -- 172.21.15.94:0/3068275417 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8032810 con 0x7fbe041608b0 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/3068275417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 msgr2=0x7fbe041647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 --2- 172.21.15.94:0/3068275417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe041647f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80097a0 tx=0x7fbdf802a180 comp rx=0 tx=0).stop 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/3068275417 shutdown_connections 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 --2- 172.21.15.94:0/3068275417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe041647f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 --2- 172.21.15.94:0/3068275417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe040d4a80 0x7fbe041642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/3068275417 >> 172.21.15.94:0/3068275417 conn(0x7fbe040d6a00 msgr2=0x7fbe040b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/3068275417 shutdown_connections 2024-04-25T16:27:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.088+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/3068275417 wait complete. 2024-04-25T16:27:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.090+0000 7fbe0a1b7640 1 Processor -- start 2024-04-25T16:27:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.090+0000 7fbe0a1b7640 1 -- start start 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe0a1b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe040d4a80 0x7fbe04152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe0a1b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe04155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe0a1b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe04164d30 0x7fbe041535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe0a1b7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe040b5950 con 0x7fbe041608b0 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe0a1b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe040d5b00 con 0x7fbe040d4a80 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe0a1b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe040b5ad0 con 0x7fbe04164d30 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe02ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe04155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe02ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe04155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42550/0 (socket says 172.21.15.94:42550) 2024-04-25T16:27:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe02ffd640 1 -- 172.21.15.94:0/1098981199 learned_addr learned my addr 172.21.15.94:0/1098981199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe037fe640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe040d4a80 0x7fbe04152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.091+0000 7fbe03fff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe04164d30 0x7fbe041535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe02ffd640 1 -- 172.21.15.94:0/1098981199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe04164d30 msgr2=0x7fbe041535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe02ffd640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe04164d30 0x7fbe041535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe02ffd640 1 -- 172.21.15.94:0/1098981199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe040d4a80 msgr2=0x7fbe04152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe02ffd640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe040d4a80 0x7fbe04152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe02ffd640 1 -- 172.21.15.94:0/1098981199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe04152ee0 con 0x7fbe041608b0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe02ffd640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe04155dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80098c0 tx=0x7fbdf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8037040 con 0x7fbe041608b0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf8042070 con 0x7fbe041608b0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8030940 con 0x7fbe041608b0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe04160fa0 con 0x7fbe041608b0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.092+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe04161560 con 0x7fbe041608b0 2024-04-25T16:27:55.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.093+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe0406a180 con 0x7fbe041608b0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbdf8030ae0 con 0x7fbe041608b0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbdebfff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbdf00744e0 0x7fbdf00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbdf80b6ce0 con 0x7fbe041608b0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbe037fe640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbdf00744e0 0x7fbdf00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbdebfff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 0x7fbdf0080430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 --> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdf804ee80 con 0x7fbdf007df40 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.094+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdf80b7170 con 0x7fbe041608b0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.095+0000 7fbe03fff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 0x7fbdf0080430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.095+0000 7fbe037fe640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbdf00744e0 0x7fbdf00769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe040b9530 tx=0x7fbdf4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.095+0000 7fbe03fff640 1 -- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 msgr2=0x7fbdf0080430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.095+0000 7fbe03fff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 0x7fbdf0080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.096+0000 7fbe03fff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 0x7fbdf0080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fbdf804ee80 command_reply(tid 1: 0 ) 2024-04-25T16:27:55.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.540+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== osd.0 v2:172.21.15.94:6809/2433651201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbdf804ee80 con 0x7fbdf007df40 2024-04-25T16:27:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.553+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 --> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbe040d4630 con 0x7fbdf007df40 2024-04-25T16:27:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:55.554+0000 7fbe03fff640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 0x7fbdf0080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fbe040d4630 command_reply(tid 2: 0 ) 2024-04-25T16:27:59.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbdebfff640 1 -- 172.21.15.94:0/1098981199 <== osd.0 v2:172.21.15.94:6809/2433651201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbe040d4630 con 0x7fbdf007df40 2024-04-25T16:27:59.015 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:27:59.015 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:27:59.015 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:27:59.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 msgr2=0x7fbdf0080430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6809/2433651201,v1:172.21.15.94:6811/2433651201] conn(0x7fbdf007df40 0x7fbdf0080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbdf00744e0 msgr2=0x7fbdf00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbdf00744e0 0x7fbdf00769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe040b9530 tx=0x7fbdf4009190 comp rx=0 tx=0).stop 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 msgr2=0x7fbe04155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 --2- 172.21.15.94:0/1098981199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe041608b0 0x7fbe04155dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80098c0 tx=0x7fbdf80046c0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe03fff640 1 -- 172.21.15.94:0/1098981199 reap_dead start 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 shutdown_connections 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.015+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 >> 172.21.15.94:0/1098981199 conn(0x7fbe040d6a00 msgr2=0x7fbe0406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.016+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 shutdown_connections 2024-04-25T16:27:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.016+0000 7fbe0a1b7640 1 -- 172.21.15.94:0/1098981199 wait complete. 2024-04-25T16:27:59.030 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T16:27:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.147+0000 7f6fd33b7640 1 Processor -- start 2024-04-25T16:27:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.147+0000 7f6fd33b7640 1 -- start start 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.148+0000 7f6fd33b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fcc0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.148+0000 7f6fd33b7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0b5570 con 0x7f6fcc0b7520 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.148+0000 7f6fd33b7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0d2b00 con 0x7f6fcc0d20d0 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.148+0000 7f6fd33b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0b5320 con 0x7f6fcc0b95f0 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.148+0000 7f6fd1954640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6fcc0b7520 0x7f6fcc162740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48048/0 (socket says 172.21.15.94:48048) 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.148+0000 7f6fd1954640 1 -- 172.21.15.94:0/1193361872 learned_addr learned my addr 172.21.15.94:0/1193361872 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1153640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fcc0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297604604 0 0) 0x7f6fcc0b5570 con 0x7f6fcc0b7520 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fb8003520 con 0x7f6fcc0b7520 2024-04-25T16:27:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 800283392 0 0) 0x7f6fb8003520 con 0x7f6fcc0b7520 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 msgr2=0x7f6fcc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fcc0cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 >> v1:172.21.15.138:6789/0 conn(0x7f6fcc0d20d0 legacy=0x7f6fcc0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0d2de0 con 0x7f6fcc0b7520 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f6fc40019f0 con 0x7f6fcc0b7520 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1153640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fcc0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 >> v1:172.21.15.94:6789/0 conn(0x7f6fcc0b7520 legacy=0x7f6fcc162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb80039e0 0x7f6fb8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb8004d40 0x7f6fb8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0b5320 con 0x7f6fb8004d40 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb800ba70 con 0x7f6fb80039e0 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb800bc20 con 0x7f6fcc0b95f0 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1954640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb8004d40 0x7f6fb8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1153640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb80039e0 0x7f6fb8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd0952640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1954640 1 -- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 msgr2=0x7f6fb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1954640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fcc0b95f0 0x7f6fb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1954640 1 -- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb80039e0 msgr2=0x7f6fb8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.149+0000 7f6fd1954640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb80039e0 0x7f6fb8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.150+0000 7f6fd1954640 1 -- 172.21.15.94:0/1193361872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb8003520 con 0x7f6fb8004d40 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.150+0000 7f6fd1954640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb8004d40 0x7f6fb8005150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400a7e0 tx=0x7f6fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecab8e793382dee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.150+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400a430 con 0x7f6fb8004d40 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.150+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc402f070 con 0x7f6fb8004d40 2024-04-25T16:27:59.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.150+0000 7f6fc3fff640 1 -- 172.21.15.94:0/1193361872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402bc10 con 0x7f6fb8004d40 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd33b7640 1 -- 172.21.15.94:0/1193361872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb8004d40 msgr2=0x7f6fb8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/1193361872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb8004d40 0x7f6fb8005150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400a7e0 tx=0x7f6fb8005830 comp rx=0 tx=0).stop 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd1954640 1 -- 172.21.15.94:0/1193361872 reap_dead start 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd33b7640 1 -- 172.21.15.94:0/1193361872 shutdown_connections 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd33b7640 1 -- 172.21.15.94:0/1193361872 >> 172.21.15.94:0/1193361872 conn(0x7f6fcc0b5ab0 msgr2=0x7f6fcc0b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd33b7640 1 -- 172.21.15.94:0/1193361872 shutdown_connections 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.151+0000 7f6fd33b7640 1 -- 172.21.15.94:0/1193361872 wait complete. 2024-04-25T16:27:59.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.153+0000 7f6fd33b7640 1 Processor -- start 2024-04-25T16:27:59.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.153+0000 7f6fd33b7640 1 -- start start 2024-04-25T16:27:59.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd33b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fb8004d40 0x7f6fcc160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd33b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc15fc10 0x7f6fcc14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd33b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 0x7f6fcc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd33b7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0b4e10 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd33b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0cf670 con 0x7f6fcc15fc10 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd33b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0b4f90 con 0x7f6fb8004d40 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd1954640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 0x7f6fcc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd1954640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 0x7f6fcc0d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42574/0 (socket says 172.21.15.94:42574) 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.154+0000 7f6fd1954640 1 -- 172.21.15.94:0/2693365554 learned_addr learned my addr 172.21.15.94:0/2693365554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1153640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fb8004d40 0x7f6fcc160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd0952640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc15fc10 0x7f6fcc14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1954640 1 -- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fb8004d40 msgr2=0x7f6fcc160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1954640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fb8004d40 0x7f6fcc160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1954640 1 -- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc15fc10 msgr2=0x7f6fcc14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1954640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc15fc10 0x7f6fcc14c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1954640 1 -- 172.21.15.94:0/2693365554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc14bda0 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd1954640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 0x7f6fcc0d0cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400a900 tx=0x7f6fc4001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400a430 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc402bc00 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4034470 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc161400 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.155+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc1616e0 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.156+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc0d2920 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.157+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6fc4046060 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.157+0000 7f6fc1ffb640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6fb00744b0 0x7f6fb00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.158+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6fc40b54d0 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.158+0000 7f6fd1153640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6fb00744b0 0x7f6fb00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.158+0000 7f6fd1153640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6fb00744b0 0x7f6fb00769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc074a30 tx=0x7f6fc8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.162+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fcc0d2920 con 0x7f6fcc0d08e0 2024-04-25T16:27:59.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.336+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f6fcc161190 con 0x7f6fb00744b0 2024-04-25T16:27:59.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.336+0000 7f6fc1ffb640 1 -- 172.21.15.94:0/2693365554 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6fcc161190 con 0x7f6fb00744b0 2024-04-25T16:27:59.337 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T16:27:59.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6fb00744b0 msgr2=0x7f6fb00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6fb00744b0 0x7f6fb00769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc074a30 tx=0x7f6fc8009190 comp rx=0 tx=0).stop 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 msgr2=0x7f6fcc0d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 0x7f6fcc0d0cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400a900 tx=0x7f6fc4001d40 comp rx=0 tx=0).stop 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 shutdown_connections 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f6fb00744b0 0x7f6fb00769c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fcc0d08e0 0x7f6fcc0d0cf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc15fc10 0x7f6fcc14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 --2- 172.21.15.94:0/2693365554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fb8004d40 0x7f6fcc160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 >> 172.21.15.94:0/2693365554 conn(0x7f6fcc0b5ab0 msgr2=0x7f6fcc071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 shutdown_connections 2024-04-25T16:27:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.339+0000 7f6fd33b7640 1 -- 172.21.15.94:0/2693365554 wait complete. 2024-04-25T16:27:59.350 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T16:27:59.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:27:59.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.462+0000 7f35b1506640 1 Processor -- start 2024-04-25T16:27:59.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.463+0000 7f35b1506640 1 -- start start 2024-04-25T16:27:59.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35b1506640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac164160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35b1506640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f35ac1646a0 0x7f35ac164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35b1506640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0b9650 con 0x7f35ac0d4c00 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35b1506640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac164ff0 con 0x7f35ac1646a0 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35b1506640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0b5660 con 0x7f35ac0b5110 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35aa7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f35ac1646a0 0x7f35ac164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35ab7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f35ac0d4c00 0x7f35ac0d5010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48060/0 (socket says 172.21.15.94:48060) 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35ab7fe640 1 -- 172.21.15.94:0/2753867337 learned_addr learned my addr 172.21.15.94:0/2753867337 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35aaffd640 1 --2- 172.21.15.94:0/2753867337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35aa7fc640 1 -- 172.21.15.94:0/2753867337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 msgr2=0x7f35ac164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35aa7fc640 1 --2- 172.21.15.94:0/2753867337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac164160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35aa7fc640 1 -- 172.21.15.94:0/2753867337 >> v1:172.21.15.94:6789/0 conn(0x7f35ac0d4c00 legacy=0x7f35ac0d5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35aa7fc640 1 -- 172.21.15.94:0/2753867337 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0d0d30 con 0x7f35ac1646a0 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.464+0000 7f35a9ffb640 1 -- 172.21.15.94:0/2753867337 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235257437 0 0) 0x7f359c001490 con 0x7f35ac0d4c00 2024-04-25T16:27:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.465+0000 7f35aa7fc640 1 --2- 172.21.15.94:0/2753867337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac1646a0 0x7f35ac164ab0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f35a00097a0 tx=0x7f35a002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7e9641e37d90bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.465+0000 7f35a9ffb640 1 -- 172.21.15.94:0/2753867337 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0037070 con 0x7f35ac1646a0 2024-04-25T16:27:59.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.465+0000 7f35a9ffb640 1 -- 172.21.15.94:0/2753867337 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a002adb0 con 0x7f35ac1646a0 2024-04-25T16:27:59.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.465+0000 7f35a9ffb640 1 -- 172.21.15.94:0/2753867337 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0032660 con 0x7f35ac1646a0 2024-04-25T16:27:59.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 -- 172.21.15.94:0/2753867337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac1646a0 msgr2=0x7f35ac164ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 --2- 172.21.15.94:0/2753867337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac1646a0 0x7f35ac164ab0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f35a00097a0 tx=0x7f35a002a060 comp rx=0 tx=0).stop 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 -- 172.21.15.94:0/2753867337 shutdown_connections 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 --2- 172.21.15.94:0/2753867337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac1646a0 0x7f35ac164ab0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 --2- 172.21.15.94:0/2753867337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac164160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 -- 172.21.15.94:0/2753867337 >> 172.21.15.94:0/2753867337 conn(0x7f35ac0b7c70 msgr2=0x7f35ac0b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 -- 172.21.15.94:0/2753867337 shutdown_connections 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.466+0000 7f35b1506640 1 -- 172.21.15.94:0/2753867337 wait complete. 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.468+0000 7f35b1506640 1 Processor -- start 2024-04-25T16:27:59.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.468+0000 7f35b1506640 1 -- start start 2024-04-25T16:27:59.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35b1506640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35b1506640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac0d4c00 0x7f35ac1fa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35b1506640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35ac1646a0 0x7f35ac1fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35b1506640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac06c850 con 0x7f35ac0d4c00 2024-04-25T16:27:59.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35b1506640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac06c570 con 0x7f35ac1646a0 2024-04-25T16:27:59.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35b1506640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0b5940 con 0x7f35ac0b5110 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35aa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac0d4c00 0x7f35ac1fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35aa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac0d4c00 0x7f35ac1fa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42592/0 (socket says 172.21.15.94:42592) 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35aa7fc640 1 -- 172.21.15.94:0/4128819097 learned_addr learned my addr 172.21.15.94:0/4128819097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.469+0000 7f35aaffd640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35ab7fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35ac1646a0 0x7f35ac1fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35aa7fc640 1 -- 172.21.15.94:0/4128819097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 msgr2=0x7f35ac0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35aa7fc640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35ac0b5110 0x7f35ac0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35aa7fc640 1 -- 172.21.15.94:0/4128819097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35ac1646a0 msgr2=0x7f35ac1fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35aa7fc640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35ac1646a0 0x7f35ac1fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35aa7fc640 1 -- 172.21.15.94:0/4128819097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0ce6e0 con 0x7f35ac0d4c00 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35aa7fc640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac0d4c00 0x7f35ac1fa290 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000c00 tx=0x7f35a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0037040 con 0x7f35ac0d4c00 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a0004030 con 0x7f35ac0d4c00 2024-04-25T16:27:59.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac0c9930 con 0x7f35ac0d4c00 2024-04-25T16:27:59.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0004400 con 0x7f35ac0d4c00 2024-04-25T16:27:59.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.470+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac0c9ef0 con 0x7f35ac0d4c00 2024-04-25T16:27:59.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.471+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35ac0ca2d0 con 0x7f35ac0d4c00 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f35a0003610 con 0x7f35ac0d4c00 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35937fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f35980744b0 0x7f35980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35aaffd640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f35980744b0 0x7f35980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f35a00b61c0 con 0x7f35ac0d4c00 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35937fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 0x7f3598080400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 --> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35a0007e50 con 0x7f359807df10 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35a00bd050 con 0x7f35ac0d4c00 2024-04-25T16:27:59.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35ab7fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 0x7f3598080400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:27:59.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.472+0000 7f35aaffd640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f35980744b0 0x7f35980769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009aa0 tx=0x7f35a4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.473+0000 7f35ab7fe640 1 -- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 msgr2=0x7f3598080400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:27:59.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.473+0000 7f35ab7fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 0x7f3598080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:27:59.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.473+0000 7f35ab7fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 0x7f3598080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f35a0007e50 command_reply(tid 1: 0 ) 2024-04-25T16:27:59.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.917+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== osd.1 v2:172.21.15.94:6800/816278269 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35a0007e50 con 0x7f359807df10 2024-04-25T16:27:59.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.930+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 --> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f35ac06a180 con 0x7f359807df10 2024-04-25T16:27:59.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:27:59.931+0000 7f35ab7fe640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 0x7f3598080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f35ac06a180 command_reply(tid 2: 0 ) 2024-04-25T16:28:03.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35937fe640 1 -- 172.21.15.94:0/4128819097 <== osd.1 v2:172.21.15.94:6800/816278269 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f35ac06a180 con 0x7f359807df10 2024-04-25T16:28:03.392 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:03.392 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:03.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 msgr2=0x7f3598080400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6800/816278269,v1:172.21.15.94:6801/816278269] conn(0x7f359807df10 0x7f3598080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f35980744b0 msgr2=0x7f35980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f35980744b0 0x7f35980769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009aa0 tx=0x7f35a4009190 comp rx=0 tx=0).stop 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac0d4c00 msgr2=0x7f35ac1fa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 --2- 172.21.15.94:0/4128819097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac0d4c00 0x7f35ac1fa290 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000c00 tx=0x7f35a00046c0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35ab7fe640 1 -- 172.21.15.94:0/4128819097 reap_dead start 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 shutdown_connections 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 >> 172.21.15.94:0/4128819097 conn(0x7f35ac0b7c70 msgr2=0x7f35ac06d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 shutdown_connections 2024-04-25T16:28:03.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.392+0000 7f35b1506640 1 -- 172.21.15.94:0/4128819097 wait complete. 2024-04-25T16:28:03.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T16:28:03.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.524+0000 7f3477684640 1 Processor -- start 2024-04-25T16:28:03.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.524+0000 7f3477684640 1 -- start start 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.525+0000 7f3477684640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347009c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.525+0000 7f3477684640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 0x7f34700cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.525+0000 7f3477684640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34700b47d0 con 0x7f34700d0250 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.525+0000 7f3477684640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f347009c950 con 0x7f34700cf900 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.525+0000 7f3477684640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34700b4210 con 0x7f3470160ab0 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3474c1f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 0x7f34700cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3475420640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347009c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3474c1f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 0x7f34700cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38056/0 (socket says 172.21.15.94:38056) 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3475420640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347009c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:45172/0 (socket says 172.21.15.94:45172) 2024-04-25T16:28:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3474c1f640 1 -- 172.21.15.94:0/1215160099 learned_addr learned my addr 172.21.15.94:0/1215160099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3475420640 1 -- 172.21.15.94:0/1215160099 >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 msgr2=0x7f34700cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3475420640 1 --2- 172.21.15.94:0/1215160099 >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 0x7f34700cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3475420640 1 -- 172.21.15.94:0/1215160099 >> v1:172.21.15.138:6789/0 conn(0x7f34700d0250 legacy=0x7f34700d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3475420640 1 -- 172.21.15.94:0/1215160099 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34700b44f0 con 0x7f3470160ab0 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.526+0000 7f3474c1f640 1 --2- 172.21.15.94:0/1215160099 >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 0x7f34700cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.527+0000 7f3475420640 1 --2- 172.21.15.94:0/1215160099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347009c370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f346c009790 tx=0x7f346c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c061e00031ee15d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.527+0000 7f3467fff640 1 -- 172.21.15.94:0/1215160099 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c037070 con 0x7f3470160ab0 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.527+0000 7f3467fff640 1 -- 172.21.15.94:0/1215160099 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c02ae50 con 0x7f3470160ab0 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.527+0000 7f3467fff640 1 -- 172.21.15.94:0/1215160099 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c032660 con 0x7f3470160ab0 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 -- 172.21.15.94:0/1215160099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 msgr2=0x7f347009c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 --2- 172.21.15.94:0/1215160099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347009c370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f346c009790 tx=0x7f346c02a100 comp rx=0 tx=0).stop 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 -- 172.21.15.94:0/1215160099 shutdown_connections 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 --2- 172.21.15.94:0/1215160099 >> v2:172.21.15.138:3300/0 conn(0x7f34700cf900 0x7f34700cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 --2- 172.21.15.94:0/1215160099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347009c370 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 -- 172.21.15.94:0/1215160099 >> 172.21.15.94:0/1215160099 conn(0x7f34700b7b90 msgr2=0x7f34700b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 -- 172.21.15.94:0/1215160099 shutdown_connections 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.528+0000 7f3477684640 1 -- 172.21.15.94:0/1215160099 wait complete. 2024-04-25T16:28:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.529+0000 7f3477684640 1 Processor -- start 2024-04-25T16:28:03.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.530+0000 7f3477684640 1 -- start start 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3477684640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34700cf900 0x7f34701f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3477684640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 0x7f34700b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3477684640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3477684640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34700d0660 con 0x7f34700d0250 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3477684640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f347006f840 con 0x7f34700cf900 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3477684640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34700b36b0 con 0x7f3470160ab0 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 0x7f34700b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 0x7f34700b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45836/0 (socket says 172.21.15.94:45836) 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 -- 172.21.15.94:0/645478371 learned_addr learned my addr 172.21.15.94:0/645478371 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3475420640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34700cf900 0x7f34701f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3475c21640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 -- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 msgr2=0x7f347006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 -- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34700cf900 msgr2=0x7f34701f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34700cf900 0x7f34701f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 -- 172.21.15.94:0/645478371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3470157ea0 con 0x7f34700d0250 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.531+0000 7f3474c1f640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 0x7f34700b07d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9e0 tx=0x7f346000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.532+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460017070 con 0x7f34700d0250 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.532+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346000c9d0 con 0x7f34700d0250 2024-04-25T16:28:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.532+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3470158180 con 0x7f34700d0250 2024-04-25T16:28:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.532+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346000cd80 con 0x7f34700d0250 2024-04-25T16:28:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.532+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3470158740 con 0x7f34700d0250 2024-04-25T16:28:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.533+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3470160300 con 0x7f34700d0250 2024-04-25T16:28:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.534+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f346001a070 con 0x7f34700d0250 2024-04-25T16:28:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.534+0000 7f3465ffb640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34540744b0 0x7f34540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.534+0000 7f3475420640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34540744b0 0x7f34540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.534+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3460095880 con 0x7f34700d0250 2024-04-25T16:28:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.535+0000 7f3475420640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34540744b0 0x7f34540769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f346c034000 tx=0x7f346c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.539+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3470160300 con 0x7f34700d0250 2024-04-25T16:28:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.716+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f347006f290 con 0x7f34540744b0 2024-04-25T16:28:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.716+0000 7f3465ffb640 1 -- 172.21.15.94:0/645478371 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f347006f290 con 0x7f34540744b0 2024-04-25T16:28:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34540744b0 msgr2=0x7f34540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34540744b0 0x7f34540769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f346c034000 tx=0x7f346c02e000 comp rx=0 tx=0).stop 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 msgr2=0x7f34700b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 0x7f34700b07d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9e0 tx=0x7f346000eea0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 shutdown_connections 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f34540744b0 0x7f34540769c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3470160ab0 0x7f347006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700d0250 0x7f34700b07d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 --2- 172.21.15.94:0/645478371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34700cf900 0x7f34701f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 >> 172.21.15.94:0/645478371 conn(0x7f34700b7b90 msgr2=0x7f34700b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 shutdown_connections 2024-04-25T16:28:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.718+0000 7f3477684640 1 -- 172.21.15.94:0/645478371 wait complete. 2024-04-25T16:28:03.730 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T16:28:03.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:28:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.842+0000 7f1371515640 1 Processor -- start 2024-04-25T16:28:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.842+0000 7f1371515640 1 -- start start 2024-04-25T16:28:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f1371515640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164870 0x7f136c06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f1371515640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f136c164c80 0x7f136c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f1371515640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136c0b9650 con 0x7f136c0b5110 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f1371515640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f136c0b5660 con 0x7f136c164c80 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f1371515640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f136c0d0d30 con 0x7f136c164870 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f136b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f136c164c80 0x7f136c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f136affd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f136c0b5110 0x7f136c164160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42974/0 (socket says 172.21.15.94:42974) 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.843+0000 7f136affd640 1 -- 172.21.15.94:0/2690720801 learned_addr learned my addr 172.21.15.94:0/2690720801 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f1363fff640 1 --2- 172.21.15.94:0/2690720801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164870 0x7f136c06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136a7fc640 1 -- 172.21.15.94:0/2690720801 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512401426 0 0) 0x7f135c001110 con 0x7f136c0b5110 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136a7fc640 1 -- 172.21.15.94:0/2690720801 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1358003520 con 0x7f136c0b5110 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136b7fe640 1 -- 172.21.15.94:0/2690720801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164870 msgr2=0x7f136c06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136b7fe640 1 --2- 172.21.15.94:0/2690720801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164870 0x7f136c06cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136b7fe640 1 -- 172.21.15.94:0/2690720801 >> v1:172.21.15.94:6789/0 conn(0x7f136c0b5110 legacy=0x7f136c164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136b7fe640 1 -- 172.21.15.94:0/2690720801 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c0d1010 con 0x7f136c164c80 2024-04-25T16:28:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.844+0000 7f136b7fe640 1 --2- 172.21.15.94:0/2690720801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c164c80 0x7f136c06d2b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1364009d90 tx=0x7f136402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cfee50c910bdae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f136a7fc640 1 -- 172.21.15.94:0/2690720801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364007740 con 0x7f136c164c80 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f136a7fc640 1 -- 172.21.15.94:0/2690720801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13640078e0 con 0x7f136c164c80 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f136a7fc640 1 -- 172.21.15.94:0/2690720801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364007cb0 con 0x7f136c164c80 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 -- 172.21.15.94:0/2690720801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c164c80 msgr2=0x7f136c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 --2- 172.21.15.94:0/2690720801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c164c80 0x7f136c06d2b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1364009d90 tx=0x7f136402a530 comp rx=0 tx=0).stop 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 -- 172.21.15.94:0/2690720801 shutdown_connections 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 --2- 172.21.15.94:0/2690720801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c164c80 0x7f136c06d2b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 --2- 172.21.15.94:0/2690720801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164870 0x7f136c06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 -- 172.21.15.94:0/2690720801 >> 172.21.15.94:0/2690720801 conn(0x7f136c0b7c70 msgr2=0x7f136c0b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.845+0000 7f1371515640 1 -- 172.21.15.94:0/2690720801 shutdown_connections 2024-04-25T16:28:03.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.846+0000 7f1371515640 1 -- 172.21.15.94:0/2690720801 wait complete. 2024-04-25T16:28:03.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.847+0000 7f1371515640 1 Processor -- start 2024-04-25T16:28:03.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.847+0000 7f1371515640 1 -- start start 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.848+0000 7f1371515640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c0b5110 0x7f136c1634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f136c164870 0x7f136c1ed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164c80 0x7f136c1edd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f136c0b5940 con 0x7f136c0b5110 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f136c06c850 con 0x7f136c164870 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f136c1608e0 con 0x7f136c164c80 2024-04-25T16:28:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c0b5110 0x7f136c1634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c0b5110 0x7f136c1634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45868/0 (socket says 172.21.15.94:45868) 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 -- 172.21.15.94:0/425219752 learned_addr learned my addr 172.21.15.94:0/425219752 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136b7fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164c80 0x7f136c1edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1363fff640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f136c164870 0x7f136c1ed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 -- 172.21.15.94:0/425219752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164c80 msgr2=0x7f136c1edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f136c164c80 0x7f136c1edd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 -- 172.21.15.94:0/425219752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f136c164870 msgr2=0x7f136c1ed7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f136c164870 0x7f136c1ed7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 -- 172.21.15.94:0/425219752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c1fa560 con 0x7f136c0b5110 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f136affd640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c0b5110 0x7f136c1634b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c3d0 tx=0x7f135c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c017070 con 0x7f136c0b5110 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c005ce0 con 0x7f136c0b5110 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c007310 con 0x7f136c0b5110 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c1fa840 con 0x7f136c0b5110 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.849+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c1639f0 con 0x7f136c0b5110 2024-04-25T16:28:03.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.850+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f136c06a180 con 0x7f136c0b5110 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.851+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f135c01a070 con 0x7f136c0b5110 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.851+0000 7f13637fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f134c0744e0 0x7f134c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.851+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f135c0950d0 con 0x7f136c0b5110 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.851+0000 7f1363fff640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f134c0744e0 0x7f134c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f13637fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 0x7f134c080430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 --> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f135c01dec0 con 0x7f134c07df40 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f135c095560 con 0x7f136c0b5110 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f136b7fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 0x7f134c080430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f1363fff640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f134c0744e0 0x7f134c0769f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1354006fd0 tx=0x7f1354008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f136b7fe640 1 -- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 msgr2=0x7f134c080430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:28:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.852+0000 7f136b7fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 0x7f134c080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:03.853+0000 7f136b7fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 0x7f134c080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f135c01dec0 command_reply(tid 1: 0 ) 2024-04-25T16:28:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:04.297+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== osd.2 v2:172.21.15.94:6824/2599289524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f135c01dec0 con 0x7f134c07df40 2024-04-25T16:28:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:04.309+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 --> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f136c1ed030 con 0x7f134c07df40 2024-04-25T16:28:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:04.310+0000 7f136b7fe640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 0x7f134c080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f136c1ed030 command_reply(tid 2: 0 ) 2024-04-25T16:28:07.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f13637fe640 1 -- 172.21.15.94:0/425219752 <== osd.2 v2:172.21.15.94:6824/2599289524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f136c1ed030 con 0x7f134c07df40 2024-04-25T16:28:07.771 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:07.772 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:07.772 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 msgr2=0x7f134c080430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6824/2599289524,v1:172.21.15.94:6825/2599289524] conn(0x7f134c07df40 0x7f134c080430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:07.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f134c0744e0 msgr2=0x7f134c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f134c0744e0 0x7f134c0769f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1354006fd0 tx=0x7f1354008040 comp rx=0 tx=0).stop 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c0b5110 msgr2=0x7f136c1634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 --2- 172.21.15.94:0/425219752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f136c0b5110 0x7f136c1634b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c3d0 tx=0x7f135c00c890 comp rx=0 tx=0).stop 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f136b7fe640 1 -- 172.21.15.94:0/425219752 reap_dead start 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 shutdown_connections 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.771+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 >> 172.21.15.94:0/425219752 conn(0x7f136c0b7c70 msgr2=0x7f136c163c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.772+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 shutdown_connections 2024-04-25T16:28:07.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.772+0000 7f1371515640 1 -- 172.21.15.94:0/425219752 wait complete. 2024-04-25T16:28:07.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T16:28:07.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.903+0000 7fdfda140640 1 Processor -- start 2024-04-25T16:28:07.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.903+0000 7fdfda140640 1 -- start start 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.904+0000 7fdfda140640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd40d1720 0x7fdfd406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.904+0000 7fdfda140640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd406c9c0 con 0x7fdfd408e780 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.904+0000 7fdfda140640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd406cca0 con 0x7fdfd40d4bb0 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.904+0000 7fdfda140640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd406dfa0 con 0x7fdfd40d1720 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.904+0000 7fdfd37fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdfd408e780 0x7fdfd40b5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42978/0 (socket says 172.21.15.94:42978) 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.904+0000 7fdfd37fe640 1 -- 172.21.15.94:0/3424457062 learned_addr learned my addr 172.21.15.94:0/3424457062 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd3fff640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd40d1720 0x7fdfd406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2849388590 0 0) 0x7fdfd406c9c0 con 0x7fdfd408e780 2024-04-25T16:28:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfbc003520 con 0x7fdfd408e780 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 595448234 0 0) 0x7fdfbc003520 con 0x7fdfd408e780 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd40d1720 msgr2=0x7fdfd406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd40d1720 0x7fdfd406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 >> v1:172.21.15.138:6789/0 conn(0x7fdfd40d4bb0 legacy=0x7fdfd40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd406cf80 con 0x7fdfd408e780 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fdfc4001670 con 0x7fdfd408e780 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 >> v1:172.21.15.94:6789/0 conn(0x7fdfd408e780 legacy=0x7fdfd40b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc0039e0 0x7fdfbc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfbc004350 0x7fdfbc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc005030 0x7fdfbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd406dfa0 con 0x7fdfbc004350 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc00bf00 con 0x7fdfbc005030 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc00c100 con 0x7fdfbc0039e0 2024-04-25T16:28:07.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfbc004350 0x7fdfbc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd37fe640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc0039e0 0x7fdfbc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.905+0000 7fdfd3fff640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc005030 0x7fdfbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc0039e0 msgr2=0x7fdfbc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc0039e0 0x7fdfbc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc005030 msgr2=0x7fdfbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc005030 0x7fdfbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/3424457062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc003520 con 0x7fdfbc004350 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfbc004350 0x7fdfbc004880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8001830 tx=0x7fdfbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383d79ee372b076e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8037070 con 0x7fdfbc004350 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc802abe0 con 0x7fdfbc004350 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.906+0000 7fdfd27fc640 1 -- 172.21.15.94:0/3424457062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8001430 con 0x7fdfbc004350 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfda140640 1 -- 172.21.15.94:0/3424457062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfbc004350 msgr2=0x7fdfbc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfda140640 1 --2- 172.21.15.94:0/3424457062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfbc004350 0x7fdfbc004880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8001830 tx=0x7fdfbc005830 comp rx=0 tx=0).stop 2024-04-25T16:28:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfd3fff640 1 -- 172.21.15.94:0/3424457062 reap_dead start 2024-04-25T16:28:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfda140640 1 -- 172.21.15.94:0/3424457062 shutdown_connections 2024-04-25T16:28:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfda140640 1 -- 172.21.15.94:0/3424457062 >> 172.21.15.94:0/3424457062 conn(0x7fdfd40b7cc0 msgr2=0x7fdfd40b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfda140640 1 -- 172.21.15.94:0/3424457062 shutdown_connections 2024-04-25T16:28:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.907+0000 7fdfda140640 1 -- 172.21.15.94:0/3424457062 wait complete. 2024-04-25T16:28:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.909+0000 7fdfda140640 1 Processor -- start 2024-04-25T16:28:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.909+0000 7fdfda140640 1 -- start start 2024-04-25T16:28:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.910+0000 7fdfda140640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc004350 0x7fdfd41525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.910+0000 7fdfda140640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 0x7fdfd414cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.910+0000 7fdfda140640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd4161f60 0x7fdfd4162370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.910+0000 7fdfda140640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd406c570 con 0x7fdfd414c740 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.910+0000 7fdfda140640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd406d540 con 0x7fdfd4161f60 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.910+0000 7fdfda140640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd406c6f0 con 0x7fdfbc004350 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 0x7fdfd414cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 0x7fdfd414cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45894/0 (socket says 172.21.15.94:45894) 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/352952334 learned_addr learned my addr 172.21.15.94:0/352952334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd37fe640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc004350 0x7fdfd41525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd3fff640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd4161f60 0x7fdfd4162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc004350 msgr2=0x7fdfd41525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc004350 0x7fdfd41525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd4161f60 msgr2=0x7fdfd4162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd4161f60 0x7fdfd4162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 -- 172.21.15.94:0/352952334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41628b0 con 0x7fdfd414c740 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd3fff640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd4161f60 0x7fdfd4162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfd2ffd640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 0x7fdfd414cb50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8001920 tx=0x7fdfc8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc803f020 con 0x7fdfd414c740 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc8005270 con 0x7fdfd414c740 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd4152b00 con 0x7fdfd414c740 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8037040 con 0x7fdfd414c740 2024-04-25T16:28:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.911+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd414d790 con 0x7fdfd414c740 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.912+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd414d220 con 0x7fdfd414c740 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.913+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdfc8042020 con 0x7fdfd414c740 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.913+0000 7fdfbbfff640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdfc00744b0 0x7fdfc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.913+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdfc80b5430 con 0x7fdfd414c740 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.913+0000 7fdfd37fe640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdfc00744b0 0x7fdfc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.914+0000 7fdfd37fe640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdfc00744b0 0x7fdfc00769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007c90 tx=0x7fdfc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:07.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:07.918+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfc8049030 con 0x7fdfd414c740 2024-04-25T16:28:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.091+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdfd406a180 con 0x7fdfc00744b0 2024-04-25T16:28:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.092+0000 7fdfbbfff640 1 -- 172.21.15.94:0/352952334 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdfd406a180 con 0x7fdfc00744b0 2024-04-25T16:28:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T16:28:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.093+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdfc00744b0 msgr2=0x7fdfc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.093+0000 7fdfda140640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdfc00744b0 0x7fdfc00769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007c90 tx=0x7fdfc4009040 comp rx=0 tx=0).stop 2024-04-25T16:28:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 msgr2=0x7fdfd414cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 0x7fdfd414cb50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8001920 tx=0x7fdfc8001230 comp rx=0 tx=0).stop 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 shutdown_connections 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fdfc00744b0 0x7fdfc00769c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd4161f60 0x7fdfd4162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfd414c740 0x7fdfd414cb50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 --2- 172.21.15.94:0/352952334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfbc004350 0x7fdfd41525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 >> 172.21.15.94:0/352952334 conn(0x7fdfd40b7cc0 msgr2=0x7fdfd40b9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 shutdown_connections 2024-04-25T16:28:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.094+0000 7fdfda140640 1 -- 172.21.15.94:0/352952334 wait complete. 2024-04-25T16:28:08.105 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T16:28:08.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:28:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.216+0000 7f80d864c640 1 Processor -- start 2024-04-25T16:28:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.216+0000 7f80d864c640 1 -- start start 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d864c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 0x7f80d0162c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d864c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f80d0163180 0x7f80d0163590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d864c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d009d0e0 con 0x7f80d00b5490 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d864c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f80d0163ad0 con 0x7f80d0163180 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d864c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80d00b4a80 con 0x7f80d00b9560 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d63e8640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f80d00b5490 0x7f80d009c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:42992/0 (socket says 172.21.15.94:42992) 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d63e8640 1 -- 172.21.15.94:0/1649608035 learned_addr learned my addr 172.21.15.94:0/1649608035 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d5be7640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 0x7f80d0162c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d6be9640 1 --2- 172.21.15.94:0/1649608035 >> v2:172.21.15.138:3300/0 conn(0x7f80d0163180 0x7f80d0163590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3488185272 0 0) 0x7f80d009d0e0 con 0x7f80d00b5490 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80bc003520 con 0x7f80d00b5490 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2241182511 0 0) 0x7f80bc003520 con 0x7f80d00b5490 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 msgr2=0x7f80d0162c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 0x7f80d0162c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 >> v2:172.21.15.138:3300/0 conn(0x7f80d0163180 msgr2=0x7f80d0163590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 --2- 172.21.15.94:0/1649608035 >> v2:172.21.15.138:3300/0 conn(0x7f80d0163180 0x7f80d0163590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d009d3c0 con 0x7f80d00b5490 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f80cc001670 con 0x7f80d00b5490 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 >> v1:172.21.15.94:6789/0 conn(0x7f80d00b5490 legacy=0x7f80d009c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80bc0039b0 0x7f80bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc004800 0x7f80bc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 0x7f80bc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80d0163ad0 con 0x7f80bc0039b0 2024-04-25T16:28:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc00ba70 con 0x7f80bc004800 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc00bc20 con 0x7f80d00b9560 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d6be9640 1 --2- 172.21.15.94:0/1649608035 >> v2:172.21.15.138:3300/0 conn(0x7f80d0163180 0x7f80d0163590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.218+0000 7f80d63e8640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80bc0039b0 0x7f80bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d5be7640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc004800 0x7f80bc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d6be9640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 0x7f80bc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d63e8640 1 -- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 msgr2=0x7f80bc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d63e8640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d00b9560 0x7f80bc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d63e8640 1 -- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc004800 msgr2=0x7f80bc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d63e8640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc004800 0x7f80bc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d63e8640 1 -- 172.21.15.94:0/1649608035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc003520 con 0x7f80bc0039b0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d5be7640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc004800 0x7f80bc004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.219+0000 7f80d63e8640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80bc0039b0 0x7f80bc003e20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f80d008d440 tx=0x7f80bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4fedfb8be33d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007d50 con 0x7f80bc0039b0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc0058d0 con 0x7f80bc0039b0 2024-04-25T16:28:08.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d53e6640 1 -- 172.21.15.94:0/1649608035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc005cc0 con 0x7f80bc0039b0 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d864c640 1 -- 172.21.15.94:0/1649608035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80bc0039b0 msgr2=0x7f80bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d864c640 1 --2- 172.21.15.94:0/1649608035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80bc0039b0 0x7f80bc003e20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f80d008d440 tx=0x7f80bc005830 comp rx=0 tx=0).stop 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d6be9640 1 -- 172.21.15.94:0/1649608035 reap_dead start 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d864c640 1 -- 172.21.15.94:0/1649608035 shutdown_connections 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d864c640 1 -- 172.21.15.94:0/1649608035 >> 172.21.15.94:0/1649608035 conn(0x7f80d00b5a90 msgr2=0x7f80d00b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d864c640 1 -- 172.21.15.94:0/1649608035 shutdown_connections 2024-04-25T16:28:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.220+0000 7f80d864c640 1 -- 172.21.15.94:0/1649608035 wait complete. 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.222+0000 7f80d864c640 1 Processor -- start 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.222+0000 7f80d864c640 1 -- start start 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d864c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc0039b0 0x7f80d00cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d864c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d01618a0 0x7f80d01f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d864c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80d01f82d0 0x7f80d01f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d864c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80d009ca30 con 0x7f80d01f82d0 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d864c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80d00cfa90 con 0x7f80bc0039b0 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d864c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80d00b4e10 con 0x7f80d01618a0 2024-04-25T16:28:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.223+0000 7f80d6be9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80d01f82d0 0x7f80d01f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80d01f82d0 0x7f80d01f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45918/0 (socket says 172.21.15.94:45918) 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 -- 172.21.15.94:0/3205019266 learned_addr learned my addr 172.21.15.94:0/3205019266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d63e8640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc0039b0 0x7f80d00cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d5be7640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d01618a0 0x7f80d01f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 -- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d01618a0 msgr2=0x7f80d01f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d01618a0 0x7f80d01f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 -- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc0039b0 msgr2=0x7f80d00cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc0039b0 0x7f80d00cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 -- 172.21.15.94:0/3205019266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d00cda80 con 0x7f80d01f82d0 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d63e8640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80bc0039b0 0x7f80d00cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d5be7640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d01618a0 0x7f80d01f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d6be9640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80d01f82d0 0x7f80d01f86e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f80d00cb020 tx=0x7f80c8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003990 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c801b070 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d009e360 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003b50 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.224+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d00cd4c0 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.225+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80d006a180 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.226+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80c8005d10 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.226+0000 7f80c6ffd640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80b40744b0 0x7f80b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.226+0000 7f80d63e8640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80b40744b0 0x7f80b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.226+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f80c8096ad0 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.226+0000 7f80c6ffd640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 0x7f80b4080410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.227+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 --> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80c800be50 con 0x7f80b407df20 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.227+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80c8096f60 con 0x7f80d01f82d0 2024-04-25T16:28:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.227+0000 7f80d5be7640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 0x7f80b4080410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.227+0000 7f80d63e8640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80b40744b0 0x7f80b40769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80cc033000 tx=0x7f80cc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.227+0000 7f80d5be7640 1 -- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 msgr2=0x7f80b4080410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:28:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.227+0000 7f80d5be7640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 0x7f80b4080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.228+0000 7f80d5be7640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 0x7f80b4080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f80c800be50 command_reply(tid 1: 0 ) 2024-04-25T16:28:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.672+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== osd.3 v2:172.21.15.94:6808/1336839764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80c800be50 con 0x7f80b407df20 2024-04-25T16:28:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.684+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 --> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f80d006d010 con 0x7f80b407df20 2024-04-25T16:28:08.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:08.685+0000 7f80d5be7640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 0x7f80b4080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f80d006d010 command_reply(tid 2: 0 ) 2024-04-25T16:28:12.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80c6ffd640 1 -- 172.21.15.94:0/3205019266 <== osd.3 v2:172.21.15.94:6808/1336839764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f80d006d010 con 0x7f80b407df20 2024-04-25T16:28:12.146 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:12.146 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:12.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 msgr2=0x7f80b4080410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6808/1336839764,v1:172.21.15.94:6810/1336839764] conn(0x7f80b407df20 0x7f80b4080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80b40744b0 msgr2=0x7f80b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f80b40744b0 0x7f80b40769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80cc033000 tx=0x7f80cc009030 comp rx=0 tx=0).stop 2024-04-25T16:28:12.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80d01f82d0 msgr2=0x7f80d01f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 --2- 172.21.15.94:0/3205019266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80d01f82d0 0x7f80d01f86e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f80d00cb020 tx=0x7f80c8009f90 comp rx=0 tx=0).stop 2024-04-25T16:28:12.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d6be9640 1 -- 172.21.15.94:0/3205019266 reap_dead start 2024-04-25T16:28:12.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 shutdown_connections 2024-04-25T16:28:12.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 >> 172.21.15.94:0/3205019266 conn(0x7f80d00b5a90 msgr2=0x7f80d00b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:12.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 shutdown_connections 2024-04-25T16:28:12.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.146+0000 7f80d864c640 1 -- 172.21.15.94:0/3205019266 wait complete. 2024-04-25T16:28:12.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T16:28:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.275+0000 7fbd38a92640 1 Processor -- start 2024-04-25T16:28:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.275+0000 7fbd38a92640 1 -- start start 2024-04-25T16:28:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd38a92640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd340cf0c0 0x7fbd340b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd38a92640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbd3415e800 0x7fbd3406d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd38a92640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3415ec10 con 0x7fbd3409c5a0 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd38a92640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbd340b50b0 con 0x7fbd3415e800 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd38a92640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbd340b5350 con 0x7fbd340cf0c0 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbd3415e800 0x7fbd3406d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbd3415e800 0x7fbd3406d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59404/0 (socket says 172.21.15.94:59404) 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 -- 172.21.15.94:0/4146612739 learned_addr learned my addr 172.21.15.94:0/4146612739 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd31d9b640 1 --2- 172.21.15.94:0/4146612739 >> v2:172.21.15.138:3300/0 conn(0x7fbd340cf0c0 0x7fbd340b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 -- 172.21.15.94:0/4146612739 >> v2:172.21.15.138:3300/0 conn(0x7fbd340cf0c0 msgr2=0x7fbd340b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 --2- 172.21.15.94:0/4146612739 >> v2:172.21.15.138:3300/0 conn(0x7fbd340cf0c0 0x7fbd340b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 -- 172.21.15.94:0/4146612739 >> v1:172.21.15.138:6789/0 conn(0x7fbd3409c5a0 legacy=0x7fbd340ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.277+0000 7fbd32d9d640 1 -- 172.21.15.94:0/4146612739 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3406cff0 con 0x7fbd3415e800 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.278+0000 7fbd32d9d640 1 --2- 172.21.15.94:0/4146612739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3415e800 0x7fbd3406d7f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd3406f990 tx=0x7fbd2c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cdcbd392cdf2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.278+0000 7fbd3159a640 1 -- 172.21.15.94:0/4146612739 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c037070 con 0x7fbd3415e800 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.278+0000 7fbd3159a640 1 -- 172.21.15.94:0/4146612739 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c02ae50 con 0x7fbd3415e800 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.278+0000 7fbd3159a640 1 -- 172.21.15.94:0/4146612739 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c032860 con 0x7fbd3415e800 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 -- 172.21.15.94:0/4146612739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3415e800 msgr2=0x7fbd3406d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 --2- 172.21.15.94:0/4146612739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3415e800 0x7fbd3406d7f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd3406f990 tx=0x7fbd2c02a300 comp rx=0 tx=0).stop 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 -- 172.21.15.94:0/4146612739 shutdown_connections 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 --2- 172.21.15.94:0/4146612739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3415e800 0x7fbd3406d7f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 --2- 172.21.15.94:0/4146612739 >> v2:172.21.15.138:3300/0 conn(0x7fbd340cf0c0 0x7fbd340b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 -- 172.21.15.94:0/4146612739 >> 172.21.15.94:0/4146612739 conn(0x7fbd340b7c70 msgr2=0x7fbd340b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 -- 172.21.15.94:0/4146612739 shutdown_connections 2024-04-25T16:28:12.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.279+0000 7fbd38a92640 1 -- 172.21.15.94:0/4146612739 wait complete. 2024-04-25T16:28:12.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.281+0000 7fbd38a92640 1 Processor -- start 2024-04-25T16:28:12.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.281+0000 7fbd38a92640 1 -- start start 2024-04-25T16:28:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.282+0000 7fbd38a92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 0x7fbd34161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.282+0000 7fbd38a92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd340cf0c0 0x7fbd340cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.282+0000 7fbd38a92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd3415e800 0x7fbd34159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.282+0000 7fbd38a92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3406dd30 con 0x7fbd3409c5a0 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.282+0000 7fbd38a92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340d0710 con 0x7fbd3415e800 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.282+0000 7fbd38a92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3406deb0 con 0x7fbd340cf0c0 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 0x7fbd34161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 0x7fbd34161430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59412/0 (socket says 172.21.15.94:59412) 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 -- 172.21.15.94:0/3393967059 learned_addr learned my addr 172.21.15.94:0/3393967059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd31d9b640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd340cf0c0 0x7fbd340cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd32d9d640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd3415e800 0x7fbd34159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 -- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd340cf0c0 msgr2=0x7fbd340cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd340cf0c0 0x7fbd340cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 -- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd3415e800 msgr2=0x7fbd34159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd3415e800 0x7fbd34159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 -- 172.21.15.94:0/3393967059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd340ca380 con 0x7fbd3409c5a0 2024-04-25T16:28:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd3259c640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 0x7fbd34161430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c730 tx=0x7fbd2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24016020 con 0x7fbd3409c5a0 2024-04-25T16:28:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd24006d50 con 0x7fbd3409c5a0 2024-04-25T16:28:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd340cc810 con 0x7fbd3409c5a0 2024-04-25T16:28:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24003bb0 con 0x7fbd3409c5a0 2024-04-25T16:28:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.283+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd340ccdd0 con 0x7fbd3409c5a0 2024-04-25T16:28:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.284+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3406a180 con 0x7fbd3409c5a0 2024-04-25T16:28:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.285+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd2400a850 con 0x7fbd3409c5a0 2024-04-25T16:28:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.285+0000 7fbd22ffd640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbd140744b0 0x7fbd140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.285+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbd240951c0 con 0x7fbd3409c5a0 2024-04-25T16:28:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.285+0000 7fbd31d9b640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbd140744b0 0x7fbd140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.286+0000 7fbd31d9b640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbd140744b0 0x7fbd140769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006fd0 tx=0x7fbd28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.290+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd24020020 con 0x7fbd3409c5a0 2024-04-25T16:28:12.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.465+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fbd3415d210 con 0x7fbd140744b0 2024-04-25T16:28:12.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.465+0000 7fbd22ffd640 1 -- 172.21.15.94:0/3393967059 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbd3415d210 con 0x7fbd140744b0 2024-04-25T16:28:12.466 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T16:28:12.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbd140744b0 msgr2=0x7fbd140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbd140744b0 0x7fbd140769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006fd0 tx=0x7fbd28008040 comp rx=0 tx=0).stop 2024-04-25T16:28:12.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 msgr2=0x7fbd34161430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 0x7fbd34161430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c730 tx=0x7fbd2400cbf0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 shutdown_connections 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fbd140744b0 0x7fbd140769c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd3415e800 0x7fbd34159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd340cf0c0 0x7fbd340cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 --2- 172.21.15.94:0/3393967059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd3409c5a0 0x7fbd34161430 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 >> 172.21.15.94:0/3393967059 conn(0x7fbd340b7c70 msgr2=0x7fbd3415db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 shutdown_connections 2024-04-25T16:28:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.468+0000 7fbd38a92640 1 -- 172.21.15.94:0/3393967059 wait complete. 2024-04-25T16:28:12.479 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T16:28:12.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:28:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.592+0000 7fa79d9a8640 1 Processor -- start 2024-04-25T16:28:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.592+0000 7fa79d9a8640 1 -- start start 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.593+0000 7fa79d9a8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa79809c8c0 0x7fa7980b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.593+0000 7fa79d9a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa7980b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.593+0000 7fa79d9a8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7980cfa60 con 0x7fa79809d420 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.593+0000 7fa79d9a8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7980b5980 con 0x7fa79809c8c0 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.593+0000 7fa79d9a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7980b7070 con 0x7fa7980b7500 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.593+0000 7fa796ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa79809d420 0x7fa79809c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50060/0 (socket says 172.21.15.94:50060) 2024-04-25T16:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa796ffd640 1 -- 172.21.15.94:0/2052990022 learned_addr learned my addr 172.21.15.94:0/2052990022 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375240383 0 0) 0x7fa7980cfa60 con 0x7fa79809d420 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa784003520 con 0x7fa79809d420 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa78ffff640 1 --2- 172.21.15.94:0/2052990022 >> v2:172.21.15.138:3300/0 conn(0x7fa79809c8c0 0x7fa7980b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7977fe640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa7980b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2376169715 0 0) 0x7fa784003520 con 0x7fa79809d420 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 msgr2=0x7fa7980b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa7980b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 >> v2:172.21.15.138:3300/0 conn(0x7fa79809c8c0 msgr2=0x7fa7980b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 --2- 172.21.15.94:0/2052990022 >> v2:172.21.15.138:3300/0 conn(0x7fa79809c8c0 0x7fa7980b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7980cfd40 con 0x7fa79809d420 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7fa788001670 con 0x7fa79809d420 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 >> v1:172.21.15.94:6789/0 conn(0x7fa79809d420 legacy=0x7fa79809c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7840039b0 0x7fa784003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa784004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d40 0x7fa784005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7980b5980 con 0x7fa7840039b0 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa78400b9c0 con 0x7fa784004d40 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa78400bb70 con 0x7fa7980b7500 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa796ffd640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7840039b0 0x7fa784003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa78ffff640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa784004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.594+0000 7fa7977fe640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d40 0x7fa784005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa796ffd640 1 -- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 msgr2=0x7fa784004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa796ffd640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa784004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa796ffd640 1 -- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d40 msgr2=0x7fa784005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa796ffd640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d40 0x7fa784005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa796ffd640 1 -- 172.21.15.94:0/2052990022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa784003520 con 0x7fa7840039b0 2024-04-25T16:28:12.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa78ffff640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b7500 0x7fa784004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa7977fe640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d40 0x7fa784005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa796ffd640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7840039b0 0x7fa784003e20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa78800c160 tx=0x7fa784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61d567e27c08410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788007c60 con 0x7fa7840039b0 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa788005af0 con 0x7fa7840039b0 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.595+0000 7fa7967fc640 1 -- 172.21.15.94:0/2052990022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788007070 con 0x7fa7840039b0 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa79d9a8640 1 -- 172.21.15.94:0/2052990022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7840039b0 msgr2=0x7fa784003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa79d9a8640 1 --2- 172.21.15.94:0/2052990022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7840039b0 0x7fa784003e20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa78800c160 tx=0x7fa784005830 comp rx=0 tx=0).stop 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa7977fe640 1 -- 172.21.15.94:0/2052990022 reap_dead start 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa79d9a8640 1 -- 172.21.15.94:0/2052990022 shutdown_connections 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa79d9a8640 1 -- 172.21.15.94:0/2052990022 >> 172.21.15.94:0/2052990022 conn(0x7fa7980b5b00 msgr2=0x7fa7980b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa79d9a8640 1 -- 172.21.15.94:0/2052990022 shutdown_connections 2024-04-25T16:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.596+0000 7fa79d9a8640 1 -- 172.21.15.94:0/2052990022 wait complete. 2024-04-25T16:28:12.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.598+0000 7fa79d9a8640 1 Processor -- start 2024-04-25T16:28:12.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.598+0000 7fa79d9a8640 1 -- start start 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa79d9a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7840039b0 0x7fa7980a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa79d9a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7981f36f0 0x7fa79809e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa79d9a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7981f3b00 0x7fa79809e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa79d9a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa79809d830 con 0x7fa7981f36f0 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa79d9a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7980cf730 con 0x7fa7981f3b00 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa79d9a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa79808d380 con 0x7fa7840039b0 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa78ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7981f36f0 0x7fa79809e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa78ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7981f36f0 0x7fa79809e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59434/0 (socket says 172.21.15.94:59434) 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.599+0000 7fa78ffff640 1 -- 172.21.15.94:0/1508432299 learned_addr learned my addr 172.21.15.94:0/1508432299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa7977fe640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7981f3b00 0x7fa79809e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa796ffd640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7840039b0 0x7fa7980a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa78ffff640 1 -- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7840039b0 msgr2=0x7fa7980a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa78ffff640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7840039b0 0x7fa7980a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa78ffff640 1 -- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7981f3b00 msgr2=0x7fa79809e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa78ffff640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7981f3b00 0x7fa79809e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa78ffff640 1 -- 172.21.15.94:0/1508432299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7980a1d70 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa7977fe640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7981f3b00 0x7fa79809e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa796ffd640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7840039b0 0x7fa7980a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa78ffff640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7981f36f0 0x7fa79809e1b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7800116d0 tx=0x7fa780011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78000f9c0 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78000fb60 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7980a1890 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780015460 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.600+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7981602f0 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.601+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa79806a180 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa780015600 con 0x7fa7981f36f0 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa794ff9640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7780744b0 0x7fa7780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa7800962d0 con 0x7fa7981f36f0 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa796ffd640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7780744b0 0x7fa7780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa794ff9640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 0x7fa778080410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 --> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa778080ae0 con 0x7fa77807df20 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.602+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa780096760 con 0x7fa7981f36f0 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.603+0000 7fa7977fe640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 0x7fa778080410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.603+0000 7fa796ffd640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7780744b0 0x7fa7780769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa788032000 tx=0x7fa788009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.603+0000 7fa7977fe640 1 -- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 msgr2=0x7fa778080410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.603+0000 7fa7977fe640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 0x7fa778080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:12.604+0000 7fa7977fe640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 0x7fa778080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fa778080ae0 command_reply(tid 1: 0 ) 2024-04-25T16:28:13.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:13.048+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== osd.4 v2:172.21.15.138:6816/3435025595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa778080ae0 con 0x7fa77807df20 2024-04-25T16:28:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:13.061+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 --> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa7980a1b20 con 0x7fa77807df20 2024-04-25T16:28:13.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:13.062+0000 7fa7977fe640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 0x7fa778080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7fa7980a1b20 command_reply(tid 2: 0 ) 2024-04-25T16:28:16.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.523+0000 7fa794ff9640 1 -- 172.21.15.94:0/1508432299 <== osd.4 v2:172.21.15.138:6816/3435025595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa7980a1b20 con 0x7fa77807df20 2024-04-25T16:28:16.523 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:16.524 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:16.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.523+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 msgr2=0x7fa778080410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.523+0000 7fa79d9a8640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.138:6816/3435025595,v1:172.21.15.138:6817/3435025595] conn(0x7fa77807df20 0x7fa778080410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7780744b0 msgr2=0x7fa7780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa7780744b0 0x7fa7780769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa788032000 tx=0x7fa788009030 comp rx=0 tx=0).stop 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7981f36f0 msgr2=0x7fa79809e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 --2- 172.21.15.94:0/1508432299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7981f36f0 0x7fa79809e1b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7800116d0 tx=0x7fa780011b90 comp rx=0 tx=0).stop 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa7977fe640 1 -- 172.21.15.94:0/1508432299 reap_dead start 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 shutdown_connections 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 >> 172.21.15.94:0/1508432299 conn(0x7fa7980b5b00 msgr2=0x7fa798071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 shutdown_connections 2024-04-25T16:28:16.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.524+0000 7fa79d9a8640 1 -- 172.21.15.94:0/1508432299 wait complete. 2024-04-25T16:28:16.539 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T16:28:16.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.657+0000 7f568eb00640 1 Processor -- start 2024-04-25T16:28:16.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.657+0000 7f568eb00640 1 -- start start 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568eb00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56880d4bb0 0x7f568806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568eb00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56880d1010 0x7f56880d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568eb00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56880d1960 con 0x7f568808e780 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568eb00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f568806dfa0 con 0x7f56880d1010 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568eb00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f568806c9c0 con 0x7f56880d4bb0 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568c89c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f568808e780 0x7f56880b5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50074/0 (socket says 172.21.15.94:50074) 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568c89c640 1 -- 172.21.15.94:0/3778259039 learned_addr learned my addr 172.21.15.94:0/3778259039 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f5687fff640 1 --2- 172.21.15.94:0/3778259039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56880d4bb0 0x7f568806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f568d09d640 1 --2- 172.21.15.94:0/3778259039 >> v2:172.21.15.138:3300/0 conn(0x7f56880d1010 0x7f56880d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f56877fe640 1 -- 172.21.15.94:0/3778259039 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1015061835 0 0) 0x7f5678001110 con 0x7f568808e780 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.658+0000 7f56877fe640 1 -- 172.21.15.94:0/3778259039 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5674003520 con 0x7f568808e780 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.659+0000 7f5687fff640 1 -- 172.21.15.94:0/3778259039 >> v2:172.21.15.138:3300/0 conn(0x7f56880d1010 msgr2=0x7f56880d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.659+0000 7f5687fff640 1 --2- 172.21.15.94:0/3778259039 >> v2:172.21.15.138:3300/0 conn(0x7f56880d1010 0x7f56880d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.659+0000 7f5687fff640 1 -- 172.21.15.94:0/3778259039 >> v1:172.21.15.94:6789/0 conn(0x7f568808e780 legacy=0x7f56880b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.659+0000 7f5687fff640 1 -- 172.21.15.94:0/3778259039 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568806cca0 con 0x7f56880d4bb0 2024-04-25T16:28:16.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.659+0000 7f568d09d640 1 --2- 172.21.15.94:0/3778259039 >> v2:172.21.15.138:3300/0 conn(0x7f56880d1010 0x7f56880d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:28:16.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.659+0000 7f5687fff640 1 --2- 172.21.15.94:0/3778259039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56880d4bb0 0x7f568806da60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56700097a0 tx=0x7f567002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4d0cc89bd3ff91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f56877fe640 1 -- 172.21.15.94:0/3778259039 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670037070 con 0x7f56880d4bb0 2024-04-25T16:28:16.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f56877fe640 1 -- 172.21.15.94:0/3778259039 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670032440 con 0x7f56880d4bb0 2024-04-25T16:28:16.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f56877fe640 1 -- 172.21.15.94:0/3778259039 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670032810 con 0x7f56880d4bb0 2024-04-25T16:28:16.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 -- 172.21.15.94:0/3778259039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56880d4bb0 msgr2=0x7f568806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 --2- 172.21.15.94:0/3778259039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56880d4bb0 0x7f568806da60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56700097a0 tx=0x7f567002a180 comp rx=0 tx=0).stop 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 -- 172.21.15.94:0/3778259039 shutdown_connections 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 --2- 172.21.15.94:0/3778259039 >> v2:172.21.15.138:3300/0 conn(0x7f56880d1010 0x7f56880d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 --2- 172.21.15.94:0/3778259039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56880d4bb0 0x7f568806da60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 -- 172.21.15.94:0/3778259039 >> 172.21.15.94:0/3778259039 conn(0x7f56880b7cc0 msgr2=0x7f56880b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 -- 172.21.15.94:0/3778259039 shutdown_connections 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.660+0000 7f568eb00640 1 -- 172.21.15.94:0/3778259039 wait complete. 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.662+0000 7f568eb00640 1 Processor -- start 2024-04-25T16:28:16.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.662+0000 7f568eb00640 1 -- start start 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f568808e780 0x7f5688156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 0x7f56881fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56880d4bb0 0x7f56881fa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f568806d540 con 0x7f56880d1010 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568806d260 con 0x7f56880d4bb0 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f568806c570 con 0x7f568808e780 2024-04-25T16:28:16.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 0x7f56881fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 0x7f56881fa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59442/0 (socket says 172.21.15.94:59442) 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 -- 172.21.15.94:0/2599860434 learned_addr learned my addr 172.21.15.94:0/2599860434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568c89c640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f568808e780 0x7f5688156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568d09d640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56880d4bb0 0x7f56881fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 -- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f568808e780 msgr2=0x7f5688156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f568808e780 0x7f5688156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 -- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56880d4bb0 msgr2=0x7f56881fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56880d4bb0 0x7f56881fa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 -- 172.21.15.94:0/2599860434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568815bf40 con 0x7f56880d1010 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568d09d640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56880d4bb0 0x7f56881fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f5687fff640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 0x7f56881fa3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5670035040 tx=0x7f56700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670041030 con 0x7f56880d1010 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567003d4e0 con 0x7f56880d1010 2024-04-25T16:28:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670037040 con 0x7f56880d1010 2024-04-25T16:28:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5688161040 con 0x7f56880d1010 2024-04-25T16:28:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.664+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568815bc60 con 0x7f56880d1010 2024-04-25T16:28:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.665+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568806a180 con 0x7f56880d1010 2024-04-25T16:28:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.666+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5670044070 con 0x7f56880d1010 2024-04-25T16:28:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.666+0000 7f56857fa640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f56680744b0 0x7f56680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.667+0000 7f568c89c640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f56680744b0 0x7f56680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.667+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f56700b5c40 con 0x7f56880d1010 2024-04-25T16:28:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.667+0000 7f568c89c640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f56680744b0 0x7f56680769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5678007fd0 tx=0x7f5678009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.671+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5670080070 con 0x7f56880d1010 2024-04-25T16:28:16.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.850+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f56881639f0 con 0x7f56680744b0 2024-04-25T16:28:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.850+0000 7f56857fa640 1 -- 172.21.15.94:0/2599860434 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f56881639f0 con 0x7f56680744b0 2024-04-25T16:28:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T16:28:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f56680744b0 msgr2=0x7f56680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f56680744b0 0x7f56680769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5678007fd0 tx=0x7f5678009040 comp rx=0 tx=0).stop 2024-04-25T16:28:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 msgr2=0x7f56881fa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 0x7f56881fa3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5670035040 tx=0x7f56700046c0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 shutdown_connections 2024-04-25T16:28:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f56680744b0 0x7f56680769c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56880d4bb0 0x7f56881fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56880d1010 0x7f56881fa3c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 --2- 172.21.15.94:0/2599860434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f568808e780 0x7f5688156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 >> 172.21.15.94:0/2599860434 conn(0x7f56880b7cc0 msgr2=0x7f56880b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 shutdown_connections 2024-04-25T16:28:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.853+0000 7f568eb00640 1 -- 172.21.15.94:0/2599860434 wait complete. 2024-04-25T16:28:16.864 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T16:28:16.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:28:16.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.978+0000 7f9940251640 1 Processor -- start 2024-04-25T16:28:16.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.979+0000 7f9940251640 1 -- start start 2024-04-25T16:28:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f9940251640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f99380d36f0 0x7f99380cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f9940251640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99380d1620 con 0x7f99380ce530 2024-04-25T16:28:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f9940251640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f993809d490 con 0x7f99380b9820 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f9940251640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f99380d1900 con 0x7f99380d36f0 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f993d7ec640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f99380ce530 0x7f99380ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50082/0 (socket says 172.21.15.94:50082) 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f993d7ec640 1 -- 172.21.15.94:0/265080562 learned_addr learned my addr 172.21.15.94:0/265080562 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f993e7ee640 1 --2- 172.21.15.94:0/265080562 >> v2:172.21.15.138:3300/0 conn(0x7f99380d36f0 0x7f99380cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.980+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806012588 0 0) 0x7f99380d1620 con 0x7f99380ce530 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9924003520 con 0x7f99380ce530 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1315728606 0 0) 0x7f9924003520 con 0x7f99380ce530 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 >> v2:172.21.15.138:3300/0 conn(0x7f99380d36f0 msgr2=0x7f99380cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 --2- 172.21.15.94:0/265080562 >> v2:172.21.15.138:3300/0 conn(0x7f99380d36f0 0x7f99380cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 >> v1:172.21.15.138:6789/0 conn(0x7f99380b9820 legacy=0x7f993806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993809d770 con 0x7f99380ce530 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f9934001670 con 0x7f99380ce530 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 >> v1:172.21.15.94:6789/0 conn(0x7f99380ce530 legacy=0x7f99380ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99240039b0 0x7f9924003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9924004800 0x7f9924004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924005250 0x7f992400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f992400c1b0 con 0x7f9924004800 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f992400c360 con 0x7f9924005250 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f992400c560 con 0x7f99240039b0 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993d7ec640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9924004800 0x7f9924004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993e7ee640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924005250 0x7f992400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993dfed640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99240039b0 0x7f9924003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993d7ec640 1 -- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99240039b0 msgr2=0x7f9924003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993d7ec640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99240039b0 0x7f9924003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993d7ec640 1 -- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924005250 msgr2=0x7f992400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993d7ec640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924005250 0x7f992400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993d7ec640 1 -- 172.21.15.94:0/265080562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9924003520 con 0x7f9924004800 2024-04-25T16:28:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993e7ee640 1 -- 172.21.15.94:0/265080562 reap_dead start 2024-04-25T16:28:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.981+0000 7f993dfed640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99240039b0 0x7f9924003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.982+0000 7f993d7ec640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9924004800 0x7f9924004c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99380d0930 tx=0x7f9924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2011019ff8fbfe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.982+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934007d30 con 0x7f9924004800 2024-04-25T16:28:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.982+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934005af0 con 0x7f9924004800 2024-04-25T16:28:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.982+0000 7f993cfeb640 1 -- 172.21.15.94:0/265080562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934007070 con 0x7f9924004800 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 -- 172.21.15.94:0/265080562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9924004800 msgr2=0x7f9924004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9924004800 0x7f9924004c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99380d0930 tx=0x7f9924005830 comp rx=0 tx=0).stop 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 -- 172.21.15.94:0/265080562 shutdown_connections 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 --2- 172.21.15.94:0/265080562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9924004800 0x7f9924004c10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 -- 172.21.15.94:0/265080562 >> 172.21.15.94:0/265080562 conn(0x7f99380d0960 msgr2=0x7f99380d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 -- 172.21.15.94:0/265080562 shutdown_connections 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.983+0000 7f9940251640 1 -- 172.21.15.94:0/265080562 wait complete. 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.985+0000 7f9940251640 1 Processor -- start 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.985+0000 7f9940251640 1 -- start start 2024-04-25T16:28:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f9940251640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924004800 0x7f99380a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f9940251640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993809fdf0 0x7f99380b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f9940251640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381e5480 0x7f99380b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f9940251640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99380d3c80 con 0x7f99381e5480 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f9940251640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99380d3b00 con 0x7f993809fdf0 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f9940251640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f993806e460 con 0x7f9924004800 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f993e7ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381e5480 0x7f99380b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f993e7ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381e5480 0x7f99380b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59456/0 (socket says 172.21.15.94:59456) 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f993e7ee640 1 -- 172.21.15.94:0/2243194980 learned_addr learned my addr 172.21.15.94:0/2243194980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f993dfed640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924004800 0x7f99380a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.986+0000 7f993d7ec640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993809fdf0 0x7f99380b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f993e7ee640 1 -- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924004800 msgr2=0x7f99380a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f993e7ee640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924004800 0x7f99380a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f993e7ee640 1 -- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993809fdf0 msgr2=0x7f99380b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f993e7ee640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993809fdf0 0x7f99380b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f993e7ee640 1 -- 172.21.15.94:0/2243194980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99380a0540 con 0x7f99381e5480 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f993e7ee640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381e5480 0x7f99380b2ad0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f993806fa40 tx=0x7f993000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000fe10 con 0x7f99381e5480 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993001d070 con 0x7f99381e5480 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99380a0820 con 0x7f99381e5480 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930015670 con 0x7f99381e5480 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.987+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99380a0de0 con 0x7f99381e5480 2024-04-25T16:28:16.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.988+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99380a11c0 con 0x7f99381e5480 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.989+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9930005d10 con 0x7f99381e5480 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.989+0000 7f992e7fc640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f991c0744b0 0x7f991c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.989+0000 7f993dfed640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f991c0744b0 0x7f991c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.989+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9930096a50 con 0x7f99381e5480 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.989+0000 7f992e7fc640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 0x7f991c080400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.990+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 --> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9930018e50 con 0x7f991c07df10 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.990+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9930096ee0 con 0x7f99381e5480 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.990+0000 7f993dfed640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f991c0744b0 0x7f991c0769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9928000c00 tx=0x7f9928009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.990+0000 7f993d7ec640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 0x7f991c080400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.990+0000 7f993d7ec640 1 -- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 msgr2=0x7f991c080400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:28:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.990+0000 7f993d7ec640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 0x7f991c080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:16.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:16.991+0000 7f993d7ec640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 0x7f991c080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f9930018e50 command_reply(tid 1: 0 ) 2024-04-25T16:28:17.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:17.435+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== osd.5 v2:172.21.15.138:6824/4211896878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9930018e50 con 0x7f991c07df10 2024-04-25T16:28:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:17.448+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 --> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f99380a02f0 con 0x7f991c07df10 2024-04-25T16:28:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:17.448+0000 7f993d7ec640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 0x7f991c080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f99380a02f0 command_reply(tid 2: 0 ) 2024-04-25T16:28:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f992e7fc640 1 -- 172.21.15.94:0/2243194980 <== osd.5 v2:172.21.15.138:6824/4211896878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f99380a02f0 con 0x7f991c07df10 2024-04-25T16:28:20.910 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:20.910 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 msgr2=0x7f991c080400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.138:6824/4211896878,v1:172.21.15.138:6825/4211896878] conn(0x7f991c07df10 0x7f991c080400 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f991c0744b0 msgr2=0x7f991c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f991c0744b0 0x7f991c0769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9928000c00 tx=0x7f9928009040 comp rx=0 tx=0).stop 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381e5480 msgr2=0x7f99380b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 --2- 172.21.15.94:0/2243194980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381e5480 0x7f99380b2ad0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f993806fa40 tx=0x7f993000f580 comp rx=0 tx=0).stop 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f993e7ee640 1 -- 172.21.15.94:0/2243194980 reap_dead start 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 shutdown_connections 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 >> 172.21.15.94:0/2243194980 conn(0x7f99380d0960 msgr2=0x7f99381e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:20.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 shutdown_connections 2024-04-25T16:28:20.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:20.910+0000 7f9940251640 1 -- 172.21.15.94:0/2243194980 wait complete. 2024-04-25T16:28:20.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T16:28:21.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.051+0000 7fceba7d6640 1 Processor -- start 2024-04-25T16:28:21.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.051+0000 7fceba7d6640 1 -- start start 2024-04-25T16:28:21.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceba7d6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fceb40b6920 0x7fceb406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceba7d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceba7d6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fceb406dac0 0x7fceb4160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceba7d6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fceb40b5ac0 con 0x7fceb40b6920 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceba7d6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fceb40702f0 con 0x7fceb406dac0 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceba7d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fceb406ded0 con 0x7fceb415c9c0 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fceb40b6920 0x7fceb406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fceb40b6920 0x7fceb406fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59458/0 (socket says 172.21.15.94:59458) 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 -- 172.21.15.94:0/2288255731 learned_addr learned my addr 172.21.15.94:0/2288255731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb8d73640 1 --2- 172.21.15.94:0/2288255731 >> v2:172.21.15.138:3300/0 conn(0x7fceb406dac0 0x7fceb4160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb37fe640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 -- 172.21.15.94:0/2288255731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 msgr2=0x7fceb406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 -- 172.21.15.94:0/2288255731 >> v2:172.21.15.138:3300/0 conn(0x7fceb406dac0 msgr2=0x7fceb4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 --2- 172.21.15.94:0/2288255731 >> v2:172.21.15.138:3300/0 conn(0x7fceb406dac0 0x7fceb4160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb3fff640 1 -- 172.21.15.94:0/2288255731 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb40b4eb0 con 0x7fceb40b6920 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.053+0000 7fceb37fe640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.054+0000 7fceb3fff640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb406fdb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fceb406e080 tx=0x7fcea402c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4986b20131cc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.054+0000 7fceb2ffd640 1 -- 172.21.15.94:0/2288255731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea403a070 con 0x7fceb40b6920 2024-04-25T16:28:21.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.054+0000 7fceb2ffd640 1 -- 172.21.15.94:0/2288255731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea4032480 con 0x7fceb40b6920 2024-04-25T16:28:21.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.054+0000 7fceb2ffd640 1 -- 172.21.15.94:0/2288255731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4032850 con 0x7fceb40b6920 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 -- 172.21.15.94:0/2288255731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 msgr2=0x7fceb406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb406fdb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fceb406e080 tx=0x7fcea402c600 comp rx=0 tx=0).stop 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 -- 172.21.15.94:0/2288255731 shutdown_connections 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 --2- 172.21.15.94:0/2288255731 >> v2:172.21.15.138:3300/0 conn(0x7fceb406dac0 0x7fceb4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 --2- 172.21.15.94:0/2288255731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb406fdb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 -- 172.21.15.94:0/2288255731 >> 172.21.15.94:0/2288255731 conn(0x7fceb4073500 msgr2=0x7fceb4073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 -- 172.21.15.94:0/2288255731 shutdown_connections 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.055+0000 7fceba7d6640 1 -- 172.21.15.94:0/2288255731 wait complete. 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.057+0000 7fceba7d6640 1 Processor -- start 2024-04-25T16:28:21.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.057+0000 7fceba7d6640 1 -- start start 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceba7d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb406dac0 0x7fceb409ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceba7d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceba7d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceba7d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb40b5750 con 0x7fceb40b6920 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceba7d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb40b5470 con 0x7fceb406dac0 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceba7d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fceb40b5f50 con 0x7fceb415c9c0 2024-04-25T16:28:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb414e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59462/0 (socket says 172.21.15.94:59462) 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.058+0000 7fceb37fe640 1 -- 172.21.15.94:0/3479074525 learned_addr learned my addr 172.21.15.94:0/3479074525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb8d73640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb3fff640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb406dac0 0x7fceb409ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb37fe640 1 -- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 msgr2=0x7fceb406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb37fe640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb37fe640 1 -- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb406dac0 msgr2=0x7fceb409ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb37fe640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb406dac0 0x7fceb409ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb37fe640 1 -- 172.21.15.94:0/3479074525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb409d4a0 con 0x7fceb40b6920 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb8d73640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb3fff640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb406dac0 0x7fceb409ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb37fe640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb414e750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcea800d800 tx=0x7fcea800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8004030 con 0x7fceb40b6920 2024-04-25T16:28:21.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea80041d0 con 0x7fceb40b6920 2024-04-25T16:28:21.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb409d780 con 0x7fceb40b6920 2024-04-25T16:28:21.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea800b6c0 con 0x7fceb40b6920 2024-04-25T16:28:21.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.059+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb409de10 con 0x7fceb40b6920 2024-04-25T16:28:21.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.060+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb406a180 con 0x7fceb40b6920 2024-04-25T16:28:21.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.061+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcea800b860 con 0x7fceb40b6920 2024-04-25T16:28:21.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.061+0000 7fceb0ff9640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcea00744b0 0x7fcea00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.062+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcea8095760 con 0x7fceb40b6920 2024-04-25T16:28:21.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.062+0000 7fceb3fff640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcea00744b0 0x7fcea00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.062+0000 7fceb3fff640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcea00744b0 0x7fcea00769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcea4037000 tx=0x7fcea4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.066+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcea805fc10 con 0x7fceb40b6920 2024-04-25T16:28:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.241+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fceb40d1b70 con 0x7fcea00744b0 2024-04-25T16:28:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.241+0000 7fceb0ff9640 1 -- 172.21.15.94:0/3479074525 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fceb40d1b70 con 0x7fcea00744b0 2024-04-25T16:28:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T16:28:21.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcea00744b0 msgr2=0x7fcea00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcea00744b0 0x7fcea00769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcea4037000 tx=0x7fcea4030000 comp rx=0 tx=0).stop 2024-04-25T16:28:21.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 msgr2=0x7fceb414e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb414e750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcea800d800 tx=0x7fcea800dcc0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 shutdown_connections 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fcea00744b0 0x7fcea00769c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fceb415c9c0 0x7fceb406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb40b6920 0x7fceb414e750 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 --2- 172.21.15.94:0/3479074525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb406dac0 0x7fceb409ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 >> 172.21.15.94:0/3479074525 conn(0x7fceb4073500 msgr2=0x7fceb40b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 shutdown_connections 2024-04-25T16:28:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.243+0000 7fceba7d6640 1 -- 172.21.15.94:0/3479074525 wait complete. 2024-04-25T16:28:21.254 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T16:28:21.254 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:28:21.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.365+0000 7f050b6a7640 1 Processor -- start 2024-04-25T16:28:21.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.365+0000 7f050b6a7640 1 -- start start 2024-04-25T16:28:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f050b6a7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050409c5a0 0x7f05040b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f050b6a7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05040b2fc0 con 0x7f05040d3410 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f050b6a7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05040b32a0 con 0x7f050406f450 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f050b6a7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05040b3580 con 0x7f050409c5a0 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f0509c44640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f05040d3410 0x7f05040ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50096/0 (socket says 172.21.15.94:50096) 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f0509c44640 1 -- 172.21.15.94:0/2431614468 learned_addr learned my addr 172.21.15.94:0/2431614468 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f0509443640 1 --2- 172.21.15.94:0/2431614468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050409c5a0 0x7f05040b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 469921111 0 0) 0x7f05040b2fc0 con 0x7f05040d3410 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f0003520 con 0x7f05040d3410 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797722572 0 0) 0x7f05040b32a0 con 0x7f050406f450 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05040b2fc0 con 0x7f050406f450 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 883913344 0 0) 0x7f04f0003520 con 0x7f05040d3410 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f0509443640 1 -- 172.21.15.94:0/2431614468 >> v1:172.21.15.138:6789/0 conn(0x7f050406f450 legacy=0x7f05040d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f0509443640 1 -- 172.21.15.94:0/2431614468 >> v1:172.21.15.94:6789/0 conn(0x7f05040d3410 legacy=0x7f05040ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.367+0000 7f0509443640 1 -- 172.21.15.94:0/2431614468 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05040b3860 con 0x7f050409c5a0 2024-04-25T16:28:21.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.368+0000 7f0509443640 1 --2- 172.21.15.94:0/2431614468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050409c5a0 0x7f05040b2790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f05000097d0 tx=0x7f050002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc55d2b0dba4142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.368+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500037070 con 0x7f050409c5a0 2024-04-25T16:28:21.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.368+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050002ac30 con 0x7f050409c5a0 2024-04-25T16:28:21.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.368+0000 7f04fbfff640 1 -- 172.21.15.94:0/2431614468 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0019f0 con 0x7f050409c5a0 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 -- 172.21.15.94:0/2431614468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050409c5a0 msgr2=0x7f05040b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 --2- 172.21.15.94:0/2431614468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050409c5a0 0x7f05040b2790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f05000097d0 tx=0x7f050002a1a0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 -- 172.21.15.94:0/2431614468 shutdown_connections 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 --2- 172.21.15.94:0/2431614468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050409c5a0 0x7f05040b2790 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 -- 172.21.15.94:0/2431614468 >> 172.21.15.94:0/2431614468 conn(0x7f0504074810 msgr2=0x7f05040b7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 -- 172.21.15.94:0/2431614468 shutdown_connections 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.369+0000 7f050b6a7640 1 -- 172.21.15.94:0/2431614468 wait complete. 2024-04-25T16:28:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.371+0000 7f050b6a7640 1 Processor -- start 2024-04-25T16:28:21.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.371+0000 7f050b6a7640 1 -- start start 2024-04-25T16:28:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f050b6a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050406f450 0x7f05040c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f050b6a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050409c5a0 0x7f0504156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f050b6a7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050415be20 0x7f050415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f050b6a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05040cf260 con 0x7f050406f450 2024-04-25T16:28:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f050b6a7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05040b2cd0 con 0x7f050409c5a0 2024-04-25T16:28:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f050b6a7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05040cf3e0 con 0x7f050415be20 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f0509443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050406f450 0x7f05040c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f0509443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050406f450 0x7f05040c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59476/0 (socket says 172.21.15.94:59476) 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.372+0000 7f0509443640 1 -- 172.21.15.94:0/2749698941 learned_addr learned my addr 172.21.15.94:0/2749698941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0508c42640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050409c5a0 0x7f0504156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509c44640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050415be20 0x7f050415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509443640 1 -- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050415be20 msgr2=0x7f050415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509443640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050415be20 0x7f050415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509443640 1 -- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050409c5a0 msgr2=0x7f0504156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509443640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050409c5a0 0x7f0504156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509443640 1 -- 172.21.15.94:0/2749698941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050415c770 con 0x7f050406f450 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0508c42640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050409c5a0 0x7f0504156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509c44640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f050415be20 0x7f050415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f0509443640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050406f450 0x7f05040c5940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0500035040 tx=0x7f05000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500042020 con 0x7f050406f450 2024-04-25T16:28:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500003c40 con 0x7f050406f450 2024-04-25T16:28:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0504146a00 con 0x7f050406f450 2024-04-25T16:28:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500037040 con 0x7f050406f450 2024-04-25T16:28:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.373+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041543b0 con 0x7f050406f450 2024-04-25T16:28:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.374+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f050406a180 con 0x7f050406f450 2024-04-25T16:28:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0500045020 con 0x7f050406f450 2024-04-25T16:28:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f04f9ffb640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f04e80744b0 0x7f04e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f0508c42640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f04e80744b0 0x7f04e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f05000b5c90 con 0x7f050406f450 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f04f9ffb640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 0x7f04e8080400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 --> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f050000af30 con 0x7f04e807df10 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.375+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f05000b8390 con 0x7f050406f450 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.376+0000 7f0509c44640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 0x7f04e8080400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.376+0000 7f0508c42640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f04e80744b0 0x7f04e80769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0504154c20 tx=0x7f04f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.376+0000 7f0509c44640 1 -- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 msgr2=0x7f04e8080400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.376+0000 7f0509c44640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 0x7f04e8080400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.377+0000 7f0509c44640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 0x7f04e8080400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f050000af30 command_reply(tid 1: 0 ) 2024-04-25T16:28:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.821+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== osd.6 v2:172.21.15.138:6800/62635939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f050000af30 con 0x7f04e807df10 2024-04-25T16:28:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.834+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 --> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0504146bd0 con 0x7f04e807df10 2024-04-25T16:28:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:21.835+0000 7f0509c44640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 0x7f04e8080400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f0504146bd0 command_reply(tid 2: 0 ) 2024-04-25T16:28:22.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:22.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:25.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f04f9ffb640 1 -- 172.21.15.94:0/2749698941 <== osd.6 v2:172.21.15.138:6800/62635939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0504146bd0 con 0x7f04e807df10 2024-04-25T16:28:25.296 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:25.296 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:25.296 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 msgr2=0x7f04e8080400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.138:6800/62635939,v1:172.21.15.138:6801/62635939] conn(0x7f04e807df10 0x7f04e8080400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f04e80744b0 msgr2=0x7f04e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f04e80744b0 0x7f04e80769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0504154c20 tx=0x7f04f4009040 comp rx=0 tx=0).stop 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050406f450 msgr2=0x7f05040c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 --2- 172.21.15.94:0/2749698941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050406f450 0x7f05040c5940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0500035040 tx=0x7f05000046c0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f0509c44640 1 -- 172.21.15.94:0/2749698941 reap_dead start 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 shutdown_connections 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.296+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 >> 172.21.15.94:0/2749698941 conn(0x7f0504074810 msgr2=0x7f05040ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.297+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 shutdown_connections 2024-04-25T16:28:25.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.297+0000 7f050b6a7640 1 -- 172.21.15.94:0/2749698941 wait complete. 2024-04-25T16:28:25.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T16:28:25.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.424+0000 7fe33be13640 1 Processor -- start 2024-04-25T16:28:25.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.424+0000 7fe33be13640 1 -- start start 2024-04-25T16:28:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33be13640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe33406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33be13640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe3340af380 0x7fe33406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33be13640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3340704b0 0x7fe3340708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33be13640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe3340b5e30 con 0x7fe3340704b0 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33be13640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe3340b5500 con 0x7fe3340af380 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33be13640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe3340b57e0 con 0x7fe3340aef70 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3340704b0 0x7fe3340708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3340704b0 0x7fe3340708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33390/0 (socket says 172.21.15.94:33390) 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 -- 172.21.15.94:0/379364092 learned_addr learned my addr 172.21.15.94:0/379364092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe339baf640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe33406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe3393ae640 1 --2- 172.21.15.94:0/379364092 >> v2:172.21.15.138:3300/0 conn(0x7fe3340af380 0x7fe33406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 -- 172.21.15.94:0/379364092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 msgr2=0x7fe33406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe33406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 -- 172.21.15.94:0/379364092 >> v2:172.21.15.138:3300/0 conn(0x7fe3340af380 msgr2=0x7fe33406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 --2- 172.21.15.94:0/379364092 >> v2:172.21.15.138:3300/0 conn(0x7fe3340af380 0x7fe33406dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe33a3b0640 1 -- 172.21.15.94:0/379364092 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3340b5ac0 con 0x7fe3340704b0 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.426+0000 7fe339baf640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe33406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.427+0000 7fe33a3b0640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe3340708c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe334050d80 tx=0x7fe32c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c104e5274cd46f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.427+0000 7fe338bad640 1 -- 172.21.15.94:0/379364092 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c037070 con 0x7fe3340704b0 2024-04-25T16:28:25.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.427+0000 7fe338bad640 1 -- 172.21.15.94:0/379364092 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c032440 con 0x7fe3340704b0 2024-04-25T16:28:25.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.427+0000 7fe338bad640 1 -- 172.21.15.94:0/379364092 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c032810 con 0x7fe3340704b0 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 -- 172.21.15.94:0/379364092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 msgr2=0x7fe3340708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe3340708c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe334050d80 tx=0x7fe32c02a1e0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 -- 172.21.15.94:0/379364092 shutdown_connections 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe3340708c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 --2- 172.21.15.94:0/379364092 >> v2:172.21.15.138:3300/0 conn(0x7fe3340af380 0x7fe33406dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 --2- 172.21.15.94:0/379364092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe33406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 -- 172.21.15.94:0/379364092 >> 172.21.15.94:0/379364092 conn(0x7fe334073500 msgr2=0x7fe334073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 -- 172.21.15.94:0/379364092 shutdown_connections 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.428+0000 7fe33be13640 1 -- 172.21.15.94:0/379364092 wait complete. 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.430+0000 7fe33be13640 1 Processor -- start 2024-04-25T16:28:25.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.430+0000 7fe33be13640 1 -- start start 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33be13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe33415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33be13640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe3341548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33be13640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3340af380 0x7fe3340d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33be13640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3340b62c0 con 0x7fe3340704b0 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33be13640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3340b6c20 con 0x7fe3340af380 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33be13640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe3340b6440 con 0x7fe3340aef70 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe33415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe33415fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33398/0 (socket says 172.21.15.94:33398) 2024-04-25T16:28:25.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 -- 172.21.15.94:0/877962195 learned_addr learned my addr 172.21.15.94:0/877962195 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe3393ae640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe3341548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe33a3b0640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3340af380 0x7fe3340d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 -- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 msgr2=0x7fe3341548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe3341548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 -- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3340af380 msgr2=0x7fe3340d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3340af380 0x7fe3340d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe339baf640 1 -- 172.21.15.94:0/877962195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe334154e10 con 0x7fe3340704b0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.431+0000 7fe3393ae640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe3341548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.432+0000 7fe339baf640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe33415fd60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe330007ec0 tx=0x7fe33000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.432+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330014620 con 0x7fe3340704b0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.432+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330019070 con 0x7fe3340704b0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.432+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330014a00 con 0x7fe3340704b0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.432+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3341564d0 con 0x7fe3340704b0 2024-04-25T16:28:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.432+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe334156a90 con 0x7fe3340704b0 2024-04-25T16:28:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.433+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33406a180 con 0x7fe3340704b0 2024-04-25T16:28:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.433+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe330014ba0 con 0x7fe3340704b0 2024-04-25T16:28:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.434+0000 7fe32a7fc640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3180744e0 0x7fe3180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.434+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe330095e60 con 0x7fe3340704b0 2024-04-25T16:28:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.434+0000 7fe3393ae640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3180744e0 0x7fe3180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.434+0000 7fe3393ae640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3180744e0 0x7fe3180769f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe3341474c0 tx=0x7fe324008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.438+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe334154e10 con 0x7fe3340704b0 2024-04-25T16:28:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.614+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fe334073eb0 con 0x7fe3180744e0 2024-04-25T16:28:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.614+0000 7fe32a7fc640 1 -- 172.21.15.94:0/877962195 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe334073eb0 con 0x7fe3180744e0 2024-04-25T16:28:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3180744e0 msgr2=0x7fe3180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3180744e0 0x7fe3180769f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe3341474c0 tx=0x7fe324008040 comp rx=0 tx=0).stop 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 msgr2=0x7fe33415fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe33415fd60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe330007ec0 tx=0x7fe33000e4b0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 shutdown_connections 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3180744e0 0x7fe3180769f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3340af380 0x7fe3340d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3340aef70 0x7fe3341548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 --2- 172.21.15.94:0/877962195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3340704b0 0x7fe33415fd60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 >> 172.21.15.94:0/877962195 conn(0x7fe334073500 msgr2=0x7fe3340b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 shutdown_connections 2024-04-25T16:28:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.616+0000 7fe33be13640 1 -- 172.21.15.94:0/877962195 wait complete. 2024-04-25T16:28:25.627 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T16:28:25.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T16:28:25.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.739+0000 7f9ec54e0640 1 Processor -- start 2024-04-25T16:28:25.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.739+0000 7f9ec54e0640 1 -- start start 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ec54e0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ec00b54c0 0x7f9ec0164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ec54e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0165600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ec54e0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec009d290 con 0x7f9ec009cd20 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ec54e0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec0165b40 con 0x7f9ec00b54c0 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ec54e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec00b4160 con 0x7f9ec01651f0 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebf7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebe7fc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ec00b54c0 0x7f9ec0164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebeffd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9ec009cd20 0x7f9ec009c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:50076/0 (socket says 172.21.15.94:50076) 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebeffd640 1 -- 172.21.15.94:0/1906659297 learned_addr learned my addr 172.21.15.94:0/1906659297 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebe7fc640 1 --2- 172.21.15.94:0/1906659297 >> v2:172.21.15.138:3300/0 conn(0x7f9ec00b54c0 0x7f9ec0164cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:45508/0 (socket says 172.21.15.94:45508) 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebdffb640 1 -- 172.21.15.94:0/1906659297 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820166970 0 0) 0x7f9ec009d290 con 0x7f9ec009cd20 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebdffb640 1 -- 172.21.15.94:0/1906659297 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ea8003520 con 0x7f9ec009cd20 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebe7fc640 1 -- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 msgr2=0x7f9ec0165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebe7fc640 1 --2- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0165600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebe7fc640 1 -- 172.21.15.94:0/1906659297 >> v1:172.21.15.138:6789/0 conn(0x7f9ec009cd20 legacy=0x7f9ec009c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebe7fc640 1 -- 172.21.15.94:0/1906659297 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec00b43b0 con 0x7f9ec00b54c0 2024-04-25T16:28:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.741+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0165600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:28:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.742+0000 7f9ebe7fc640 1 --2- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 0x7f9ec0164cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097a0 tx=0x7f9eb402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db51d621463e7ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.742+0000 7f9ebdffb640 1 -- 172.21.15.94:0/1906659297 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4037070 con 0x7f9ec00b54c0 2024-04-25T16:28:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.742+0000 7f9ebdffb640 1 -- 172.21.15.94:0/1906659297 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb402ae50 con 0x7f9ec00b54c0 2024-04-25T16:28:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.742+0000 7f9ebdffb640 1 -- 172.21.15.94:0/1906659297 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4032640 con 0x7f9ec00b54c0 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 -- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 msgr2=0x7f9ec0164cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 --2- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 0x7f9ec0164cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097a0 tx=0x7f9eb402a100 comp rx=0 tx=0).stop 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 -- 172.21.15.94:0/1906659297 shutdown_connections 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 --2- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0165600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 --2- 172.21.15.94:0/1906659297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 0x7f9ec0164cb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 -- 172.21.15.94:0/1906659297 >> 172.21.15.94:0/1906659297 conn(0x7f9ec00b5ae0 msgr2=0x7f9ec00b5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 -- 172.21.15.94:0/1906659297 shutdown_connections 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.743+0000 7f9ec54e0640 1 -- 172.21.15.94:0/1906659297 wait complete. 2024-04-25T16:28:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.745+0000 7f9ec54e0640 1 Processor -- start 2024-04-25T16:28:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.745+0000 7f9ec54e0640 1 -- start start 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ec54e0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec009cd20 0x7f9ec01639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ec54e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 0x7f9ec0072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ec54e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ec54e0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec009caa0 con 0x7f9ec009cd20 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ec54e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec00b4a00 con 0x7f9ec00b54c0 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ec54e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec00b5020 con 0x7f9ec01651f0 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.746+0000 7f9ebeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec009cd20 0x7f9ec01639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec009cd20 0x7f9ec01639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33404/0 (socket says 172.21.15.94:33404) 2024-04-25T16:28:25.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 -- 172.21.15.94:0/2756046572 learned_addr learned my addr 172.21.15.94:0/2756046572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebe7fc640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 0x7f9ec0072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 -- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 msgr2=0x7f9ec0072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 -- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 msgr2=0x7f9ec0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec00b54c0 0x7f9ec0072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 -- 172.21.15.94:0/2756046572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec00ce2e0 con 0x7f9ec009cd20 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ec01651f0 0x7f9ec0072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ebeffd640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec009cd20 0x7f9ec01639b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0001460 tx=0x7f9eb000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0017070 con 0x7f9ec009cd20 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb000ac00 con 0x7f9ec009cd20 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec00cb1f0 con 0x7f9ec009cd20 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0012640 con 0x7f9ec009cd20 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.747+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec00c8ee0 con 0x7f9ec009cd20 2024-04-25T16:28:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.748+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ec006a180 con 0x7f9ec009cd20 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.749+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9eb001a070 con 0x7f9ec009cd20 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.749+0000 7f9ea77fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9eac0744b0 0x7f9eac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9eb00959e0 con 0x7f9ec009cd20 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ebe7fc640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9eac0744b0 0x7f9eac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ea77fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 0x7f9eac080410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 --> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9eac080ae0 con 0x7f9eac07df20 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9eb0095e70 con 0x7f9ec009cd20 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ebe7fc640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9eac0744b0 0x7f9eac0769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4034000 tx=0x7f9eb402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 0x7f9eac080410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ebf7fe640 1 -- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 msgr2=0x7f9eac080410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-04-25T16:28:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.750+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 0x7f9eac080410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:25.751+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 0x7f9eac080410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f9eac080ae0 command_reply(tid 1: 0 ) 2024-04-25T16:28:26.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:26.195+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== osd.7 v2:172.21.15.138:6808/300369448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9eac080ae0 con 0x7f9eac07df20 2024-04-25T16:28:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:26.208+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 --> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9ec00cdd90 con 0x7f9eac07df20 2024-04-25T16:28:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:26.208+0000 7f9ebf7fe640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 0x7f9eac080410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).queue_received will delay after 1.71406e+09 on 0x7f9ec00cdd90 command_reply(tid 2: 0 ) 2024-04-25T16:28:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ea77fe640 1 -- 172.21.15.94:0/2756046572 <== osd.7 v2:172.21.15.138:6808/300369448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9ec00cdd90 con 0x7f9eac07df20 2024-04-25T16:28:29.670 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:28:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T16:28:29.670 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 msgr2=0x7f9eac080410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.138:6808/300369448,v1:172.21.15.138:6809/300369448] conn(0x7f9eac07df20 0x7f9eac080410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9eac0744b0 msgr2=0x7f9eac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f9eac0744b0 0x7f9eac0769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4034000 tx=0x7f9eb402e000 comp rx=0 tx=0).stop 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec009cd20 msgr2=0x7f9ec01639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 --2- 172.21.15.94:0/2756046572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec009cd20 0x7f9ec01639b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0001460 tx=0x7f9eb000cf70 comp rx=0 tx=0).stop 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ebf7fe640 1 -- 172.21.15.94:0/2756046572 reap_dead start 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 shutdown_connections 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 >> 172.21.15.94:0/2756046572 conn(0x7f9ec00b5ae0 msgr2=0x7f9ec009c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 shutdown_connections 2024-04-25T16:28:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.670+0000 7f9ec54e0640 1 -- 172.21.15.94:0/2756046572 wait complete. 2024-04-25T16:28:29.686 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T16:28:29.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.800+0000 7ff317442640 1 Processor -- start 2024-04-25T16:28:29.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.800+0000 7ff317442640 1 -- start start 2024-04-25T16:28:29.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff317442640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff3100aef70 0x7ff31006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:29.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff317442640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff31006dac0 0x7ff31015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:29.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff317442640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31006ded0 con 0x7ff3100af380 2024-04-25T16:28:29.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff317442640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff31015e940 con 0x7ff3100aef70 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff317442640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff3100b5e30 con 0x7ff31006dac0 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff3100aef70 0x7ff31006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff3100aef70 0x7ff31006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33410/0 (socket says 172.21.15.94:33410) 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 -- 172.21.15.94:0/343600260 learned_addr learned my addr 172.21.15.94:0/343600260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3159df640 1 --2- 172.21.15.94:0/343600260 >> v2:172.21.15.138:3300/0 conn(0x7ff31006dac0 0x7ff31015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 -- 172.21.15.94:0/343600260 >> v2:172.21.15.138:3300/0 conn(0x7ff31006dac0 msgr2=0x7ff31015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 --2- 172.21.15.94:0/343600260 >> v2:172.21.15.138:3300/0 conn(0x7ff31006dac0 0x7ff31015e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 -- 172.21.15.94:0/343600260 >> v1:172.21.15.138:6789/0 conn(0x7ff3100af380 legacy=0x7ff3100704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.802+0000 7ff3151de640 1 -- 172.21.15.94:0/343600260 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3100b5500 con 0x7ff3100aef70 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.803+0000 7ff3151de640 1 --2- 172.21.15.94:0/343600260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3100aef70 0x7ff31006d580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff3100ce380 tx=0x7ff30c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f7160641b4230f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.803+0000 7ff307fff640 1 -- 172.21.15.94:0/343600260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c036070 con 0x7ff3100aef70 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.803+0000 7ff307fff640 1 -- 172.21.15.94:0/343600260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c031600 con 0x7ff3100aef70 2024-04-25T16:28:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.803+0000 7ff307fff640 1 -- 172.21.15.94:0/343600260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0319d0 con 0x7ff3100aef70 2024-04-25T16:28:29.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 -- 172.21.15.94:0/343600260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3100aef70 msgr2=0x7ff31006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 --2- 172.21.15.94:0/343600260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3100aef70 0x7ff31006d580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff3100ce380 tx=0x7ff30c02a730 comp rx=0 tx=0).stop 2024-04-25T16:28:29.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 -- 172.21.15.94:0/343600260 shutdown_connections 2024-04-25T16:28:29.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 --2- 172.21.15.94:0/343600260 >> v2:172.21.15.138:3300/0 conn(0x7ff31006dac0 0x7ff31015e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 --2- 172.21.15.94:0/343600260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3100aef70 0x7ff31006d580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 -- 172.21.15.94:0/343600260 >> 172.21.15.94:0/343600260 conn(0x7ff310073500 msgr2=0x7ff310073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:29.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 -- 172.21.15.94:0/343600260 shutdown_connections 2024-04-25T16:28:29.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.804+0000 7ff317442640 1 -- 172.21.15.94:0/343600260 wait complete. 2024-04-25T16:28:29.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.806+0000 7ff317442640 1 Processor -- start 2024-04-25T16:28:29.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.806+0000 7ff317442640 1 -- start start 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff317442640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 0x7ff3100cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff317442640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3100aef70 0x7ff3101595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff317442640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3100af380 0x7ff310159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff317442640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3100b6a10 con 0x7ff31006dac0 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff317442640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3100b5ac0 con 0x7ff3100af380 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff317442640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3100b6b90 con 0x7ff3100aef70 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff3151de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 0x7ff3100cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff3151de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 0x7ff3100cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33420/0 (socket says 172.21.15.94:33420) 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff3151de640 1 -- 172.21.15.94:0/2975490170 learned_addr learned my addr 172.21.15.94:0/2975490170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff3159df640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3100af380 0x7ff310159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.807+0000 7ff3149dd640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3100aef70 0x7ff3101595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3151de640 1 -- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3100aef70 msgr2=0x7ff3101595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3151de640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3100aef70 0x7ff3101595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3151de640 1 -- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3100af380 msgr2=0x7ff310159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3151de640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3100af380 0x7ff310159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3151de640 1 -- 172.21.15.94:0/2975490170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3100ccd30 con 0x7ff31006dac0 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3159df640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3100af380 0x7ff310159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff3151de640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 0x7ff3100cc0b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff30c034040 tx=0x7ff30c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c036050 con 0x7ff31006dac0 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c042070 con 0x7ff31006dac0 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3100cd010 con 0x7ff31006dac0 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c009510 con 0x7ff31006dac0 2024-04-25T16:28:29.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.808+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3100cd5d0 con 0x7ff31006dac0 2024-04-25T16:28:29.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.809+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31006a180 con 0x7ff31006dac0 2024-04-25T16:28:29.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.810+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff30c007c50 con 0x7ff31006dac0 2024-04-25T16:28:29.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.810+0000 7ff305ffb640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff2f40744b0 0x7ff2f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:29.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.810+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff30c0b6430 con 0x7ff31006dac0 2024-04-25T16:28:29.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.810+0000 7ff3149dd640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff2f40744b0 0x7ff2f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:29.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.811+0000 7ff3149dd640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff2f40744b0 0x7ff2f40769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff300009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.815+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff30c080850 con 0x7ff31006dac0 2024-04-25T16:28:29.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.993+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7ff31006d8a0 con 0x7ff2f40744b0 2024-04-25T16:28:29.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.994+0000 7ff305ffb640 1 -- 172.21.15.94:0/2975490170 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff31006d8a0 con 0x7ff2f40744b0 2024-04-25T16:28:29.994 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T16:28:29.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff2f40744b0 msgr2=0x7ff2f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff2f40744b0 0x7ff2f40769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff300009040 comp rx=0 tx=0).stop 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 msgr2=0x7ff3100cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 0x7ff3100cc0b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff30c034040 tx=0x7ff30c004a40 comp rx=0 tx=0).stop 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 shutdown_connections 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7ff2f40744b0 0x7ff2f40769c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3100af380 0x7ff310159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3100aef70 0x7ff3101595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 --2- 172.21.15.94:0/2975490170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31006dac0 0x7ff3100cc0b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 >> 172.21.15.94:0/2975490170 conn(0x7ff310073500 msgr2=0x7ff3101e5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 shutdown_connections 2024-04-25T16:28:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:29.996+0000 7ff317442640 1 -- 172.21.15.94:0/2975490170 wait complete. 2024-04-25T16:28:30.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:28:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.114+0000 7f15f950d640 1 Processor -- start 2024-04-25T16:28:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.114+0000 7f15f950d640 1 -- start start 2024-04-25T16:28:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f950d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15f415d750 0x7f15f40d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f950d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f40d33c0 0x7f15f40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f950d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f40d37d0 con 0x7f15f409bc70 2024-04-25T16:28:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f950d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f15f40cd540 con 0x7f15f415d750 2024-04-25T16:28:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f950d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15f409c1e0 con 0x7f15f40d33c0 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f40d33c0 0x7f15f40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f27fc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15f415d750 0x7f15f40d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f2ffd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f15f409bc70 0x7f15f406e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:50092/0 (socket says 172.21.15.94:50092) 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.115+0000 7f15f2ffd640 1 -- 172.21.15.94:0/3425754101 learned_addr learned my addr 172.21.15.94:0/3425754101 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.116+0000 7f15f37fe640 1 -- 172.21.15.94:0/3425754101 >> v2:172.21.15.138:3300/0 conn(0x7f15f415d750 msgr2=0x7f15f40d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.116+0000 7f15f37fe640 1 --2- 172.21.15.94:0/3425754101 >> v2:172.21.15.138:3300/0 conn(0x7f15f415d750 0x7f15f40d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.116+0000 7f15f37fe640 1 -- 172.21.15.94:0/3425754101 >> v1:172.21.15.138:6789/0 conn(0x7f15f409bc70 legacy=0x7f15f406e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.116+0000 7f15f37fe640 1 -- 172.21.15.94:0/3425754101 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f40cd790 con 0x7f15f40d33c0 2024-04-25T16:28:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.116+0000 7f15f27fc640 1 --2- 172.21.15.94:0/3425754101 >> v2:172.21.15.138:3300/0 conn(0x7f15f415d750 0x7f15f40d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:28:30.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.116+0000 7f15f37fe640 1 --2- 172.21.15.94:0/3425754101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f40d33c0 0x7f15f40ce270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0097a0 tx=0x7f15ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15eec4bf85b3361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:30.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f1ffb640 1 -- 172.21.15.94:0/3425754101 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec037070 con 0x7f15f40d33c0 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f1ffb640 1 -- 172.21.15.94:0/3425754101 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec032440 con 0x7f15f40d33c0 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f1ffb640 1 -- 172.21.15.94:0/3425754101 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec032810 con 0x7f15f40d33c0 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f950d640 1 -- 172.21.15.94:0/3425754101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f40d33c0 msgr2=0x7f15f40ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f950d640 1 --2- 172.21.15.94:0/3425754101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f40d33c0 0x7f15f40ce270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0097a0 tx=0x7f15ec02a180 comp rx=0 tx=0).stop 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f950d640 1 -- 172.21.15.94:0/3425754101 shutdown_connections 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f950d640 1 --2- 172.21.15.94:0/3425754101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f40d33c0 0x7f15f40ce270 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f950d640 1 --2- 172.21.15.94:0/3425754101 >> v2:172.21.15.138:3300/0 conn(0x7f15f415d750 0x7f15f40d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.117+0000 7f15f950d640 1 -- 172.21.15.94:0/3425754101 >> 172.21.15.94:0/3425754101 conn(0x7f15f40b7b90 msgr2=0x7f15f40b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.118+0000 7f15f950d640 1 -- 172.21.15.94:0/3425754101 shutdown_connections 2024-04-25T16:28:30.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.118+0000 7f15f950d640 1 -- 172.21.15.94:0/3425754101 wait complete. 2024-04-25T16:28:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.119+0000 7f15f950d640 1 Processor -- start 2024-04-25T16:28:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.119+0000 7f15f950d640 1 -- start start 2024-04-25T16:28:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f950d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f409bc70 0x7f15f40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f950d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 0x7f15f4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f950d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f415d750 0x7f15f4157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f950d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15f40ce7b0 con 0x7f15f40d33c0 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f950d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15f40cdd50 con 0x7f15f415d750 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f950d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15f40ce930 con 0x7f15f409bc70 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 0x7f15f4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 0x7f15f4156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33428/0 (socket says 172.21.15.94:33428) 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 -- 172.21.15.94:0/3679561776 learned_addr learned my addr 172.21.15.94:0/3679561776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f2ffd640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f409bc70 0x7f15f40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f37fe640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f415d750 0x7f15f4157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 -- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f409bc70 msgr2=0x7f15f40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f409bc70 0x7f15f40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 -- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f415d750 msgr2=0x7f15f4157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:30.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f415d750 0x7f15f4157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 -- 172.21.15.94:0/3679561776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f40ca530 con 0x7f15f40d33c0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.121+0000 7f15f27fc640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 0x7f15f4156de0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15e8010e40 tx=0x7f15e800e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.122+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e801e070 con 0x7f15f40d33c0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.122+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e8011860 con 0x7f15f40d33c0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.122+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f40ca810 con 0x7f15f40d33c0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.122+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8011c30 con 0x7f15f40d33c0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.122+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f40cadd0 con 0x7f15f40d33c0 2024-04-25T16:28:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.122+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f406a180 con 0x7f15f40d33c0 2024-04-25T16:28:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.123+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15e8021070 con 0x7f15f40d33c0 2024-04-25T16:28:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.124+0000 7f15db7fe640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f15e00744b0 0x7f15e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.124+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f15e8098080 con 0x7f15f40d33c0 2024-04-25T16:28:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.124+0000 7f15f2ffd640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f15e00744b0 0x7f15e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.124+0000 7f15f2ffd640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f15e00744b0 0x7f15e00769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005c50 tx=0x7f15e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.128+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15e8061d40 con 0x7f15f40d33c0 2024-04-25T16:28:30.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.301+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15f415a8e0 con 0x7f15e00744b0 2024-04-25T16:28:30.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.309+0000 7f15db7fe640 1 -- 172.21.15.94:0/3679561776 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+331390 (secure 0 0 0) 0x7f15f415a8e0 con 0x7f15e00744b0 2024-04-25T16:28:30.313 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:28:30.314 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:28:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f15e00744b0 msgr2=0x7f15e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f15e00744b0 0x7f15e00769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005c50 tx=0x7f15e4009040 comp rx=0 tx=0).stop 2024-04-25T16:28:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 msgr2=0x7f15f4156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 0x7f15f4156de0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15e8010e40 tx=0x7f15e800e730 comp rx=0 tx=0).stop 2024-04-25T16:28:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 shutdown_connections 2024-04-25T16:28:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f15e00744b0 0x7f15e00769c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f415d750 0x7f15f4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15f40d33c0 0x7f15f4156de0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 --2- 172.21.15.94:0/3679561776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15f409bc70 0x7f15f40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 >> 172.21.15.94:0/3679561776 conn(0x7f15f40b7b90 msgr2=0x7f15f406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 shutdown_connections 2024-04-25T16:28:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:30.312+0000 7f15f950d640 1 -- 172.21.15.94:0/3679561776 wait complete. 2024-04-25T16:28:30.324 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-25T16:28:29.319650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605981,"num_objects":25,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":169,"num_write_kb":1668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236256,"kb_used_data":3488,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738464,"statfs":{"total":773094113280,"available":772852187136,"internally_reserved":0,"allocated":3571712,"data_stored":2155322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.1f","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:01.512673+0000","last_change":"2024-04-25T16:28:01.512707+0000","last_active":"2024-04-25T16:28:01.512673+0000","last_peered":"2024-04-25T16:28:01.512673+0000","last_clean":"2024-04-25T16:28:01.512673+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:28:01.512673+0000","last_undegraded":"2024-04-25T16:28:01.512673+0000","last_fullsized":"2024-04-25T16:28:01.512673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-25T16:28:01.512572+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-25T16:28:01.512572+0000","last_clean_scrub_stamp":"2024-04-25T16:28:01.512572+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:31.188129+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.671143+0000","last_change":"2024-04-25T16:28:12.671186+0000","last_active":"2024-04-25T16:28:12.671143+0000","last_peered":"2024-04-25T16:28:12.671143+0000","last_clean":"2024-04-25T16:28:12.671143+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:28:12.671143+0000","last_undegraded":"2024-04-25T16:28:12.671143+0000","last_fullsized":"2024-04-25T16:28:12.671143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.671030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.671030+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.671030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:14.544960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694282+0000","last_change":"2024-04-25T16:27:00.607533+0000","last_active":"2024-04-25T16:27:36.694282+0000","last_peered":"2024-04-25T16:27:36.694282+0000","last_clean":"2024-04-25T16:27:36.694282+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:27:36.694282+0000","last_undegraded":"2024-04-25T16:27:36.694282+0000","last_fullsized":"2024-04-25T16:27:36.694282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:18.718029+0000","last_change":"2024-04-25T16:28:18.718083+0000","last_active":"2024-04-25T16:28:18.718029+0000","last_peered":"2024-04-25T16:28:18.718029+0000","last_clean":"2024-04-25T16:28:18.718029+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:28:18.718029+0000","last_undegraded":"2024-04-25T16:28:18.718029+0000","last_fullsized":"2024-04-25T16:28:18.718029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T16:28:18.717906+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T16:28:18.717906+0000","last_clean_scrub_stamp":"2024-04-25T16:28:18.717906+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:51.102949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:59.464325+0000","last_change":"2024-04-25T16:27:59.464351+0000","last_active":"2024-04-25T16:27:59.464325+0000","last_peered":"2024-04-25T16:27:59.464325+0000","last_clean":"2024-04-25T16:27:59.464325+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:59.464325+0000","last_undegraded":"2024-04-25T16:27:59.464325+0000","last_fullsized":"2024-04-25T16:27:59.464325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:27:59.464232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:27:59.464232+0000","last_clean_scrub_stamp":"2024-04-25T16:27:59.464232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:51.004937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:25.769219+0000","last_change":"2024-04-25T16:28:25.769251+0000","last_active":"2024-04-25T16:28:25.769219+0000","last_peered":"2024-04-25T16:28:25.769219+0000","last_clean":"2024-04-25T16:28:25.769219+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:28:25.769219+0000","last_undegraded":"2024-04-25T16:28:25.769219+0000","last_fullsized":"2024-04-25T16:28:25.769219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:25.769123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:25.769123+0000","last_clean_scrub_stamp":"2024-04-25T16:28:25.769123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:56.155313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699035+0000","last_change":"2024-04-25T16:26:55.559987+0000","last_active":"2024-04-25T16:27:36.699035+0000","last_peered":"2024-04-25T16:27:36.699035+0000","last_clean":"2024-04-25T16:27:36.699035+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:27:36.699035+0000","last_undegraded":"2024-04-25T16:27:36.699035+0000","last_fullsized":"2024-04-25T16:27:36.699035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:10.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764588+0000","last_change":"2024-04-25T16:26:56.562478+0000","last_active":"2024-04-25T16:27:36.764588+0000","last_peered":"2024-04-25T16:27:36.764588+0000","last_clean":"2024-04-25T16:27:36.764588+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:27:36.764588+0000","last_undegraded":"2024-04-25T16:27:36.764588+0000","last_fullsized":"2024-04-25T16:27:36.764588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:23.849952+0000","last_change":"2024-04-25T16:28:23.849984+0000","last_active":"2024-04-25T16:28:23.849952+0000","last_peered":"2024-04-25T16:28:23.849952+0000","last_clean":"2024-04-25T16:28:23.849952+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:28:23.849952+0000","last_undegraded":"2024-04-25T16:28:23.849952+0000","last_fullsized":"2024-04-25T16:28:23.849952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:23.849835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:23.849835+0000","last_clean_scrub_stamp":"2024-04-25T16:28:23.849835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:56.903111+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697999+0000","last_change":"2024-04-25T16:26:55.561995+0000","last_active":"2024-04-25T16:27:36.697999+0000","last_peered":"2024-04-25T16:27:36.697999+0000","last_clean":"2024-04-25T16:27:36.697999+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:27:36.697999+0000","last_undegraded":"2024-04-25T16:27:36.697999+0000","last_fullsized":"2024-04-25T16:27:36.697999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:04.656191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:23.084686+0000","last_change":"2024-04-25T16:28:23.084731+0000","last_active":"2024-04-25T16:28:23.084686+0000","last_peered":"2024-04-25T16:28:23.084686+0000","last_clean":"2024-04-25T16:28:23.084686+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:28:23.084686+0000","last_undegraded":"2024-04-25T16:28:23.084686+0000","last_fullsized":"2024-04-25T16:28:23.084686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:23.084560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:23.084560+0000","last_clean_scrub_stamp":"2024-04-25T16:28:23.084560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:24.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:24.810513+0000","last_change":"2024-04-25T16:28:24.810583+0000","last_active":"2024-04-25T16:28:24.810513+0000","last_peered":"2024-04-25T16:28:24.810513+0000","last_clean":"2024-04-25T16:28:24.810513+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:28:24.810513+0000","last_undegraded":"2024-04-25T16:28:24.810513+0000","last_fullsized":"2024-04-25T16:28:24.810513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:24.810405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:24.810405+0000","last_clean_scrub_stamp":"2024-04-25T16:28:24.810405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:54.863780+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697349+0000","last_change":"2024-04-25T16:26:56.551272+0000","last_active":"2024-04-25T16:27:36.697349+0000","last_peered":"2024-04-25T16:27:36.697349+0000","last_clean":"2024-04-25T16:27:36.697349+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:27:36.697349+0000","last_undegraded":"2024-04-25T16:27:36.697349+0000","last_fullsized":"2024-04-25T16:27:36.697349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698555+0000","last_change":"2024-04-25T16:26:55.559982+0000","last_active":"2024-04-25T16:27:36.698555+0000","last_peered":"2024-04-25T16:27:36.698555+0000","last_clean":"2024-04-25T16:27:36.698555+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:27:36.698555+0000","last_undegraded":"2024-04-25T16:27:36.698555+0000","last_fullsized":"2024-04-25T16:27:36.698555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:06.851293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.751010+0000","last_change":"2024-04-25T16:27:00.591332+0000","last_active":"2024-04-25T16:27:36.751010+0000","last_peered":"2024-04-25T16:27:36.751010+0000","last_clean":"2024-04-25T16:27:36.751010+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:27:36.751010+0000","last_undegraded":"2024-04-25T16:27:36.751010+0000","last_fullsized":"2024-04-25T16:27:36.751010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:23.666726+0000","last_change":"2024-04-25T16:28:23.666752+0000","last_active":"2024-04-25T16:28:23.666726+0000","last_peered":"2024-04-25T16:28:23.666726+0000","last_clean":"2024-04-25T16:28:23.666726+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:28:23.666726+0000","last_undegraded":"2024-04-25T16:28:23.666726+0000","last_fullsized":"2024-04-25T16:28:23.666726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:23.666628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:23.666628+0000","last_clean_scrub_stamp":"2024-04-25T16:28:23.666628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:01.002536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:04.469988+0000","last_change":"2024-04-25T16:28:04.470030+0000","last_active":"2024-04-25T16:28:04.469988+0000","last_peered":"2024-04-25T16:28:04.469988+0000","last_clean":"2024-04-25T16:28:04.469988+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:28:04.469988+0000","last_undegraded":"2024-04-25T16:28:04.469988+0000","last_fullsized":"2024-04-25T16:28:04.469988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:04.469891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:04.469891+0000","last_clean_scrub_stamp":"2024-04-25T16:28:04.469891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:12.364601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.790684+0000","last_change":"2024-04-25T16:28:26.790712+0000","last_active":"2024-04-25T16:28:26.790684+0000","last_peered":"2024-04-25T16:28:26.790684+0000","last_clean":"2024-04-25T16:28:26.790684+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:28:26.790684+0000","last_undegraded":"2024-04-25T16:28:26.790684+0000","last_fullsized":"2024-04-25T16:28:26.790684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.790575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.790575+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.790575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.054670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:20.715683+0000","last_change":"2024-04-25T16:28:20.715727+0000","last_active":"2024-04-25T16:28:20.715683+0000","last_peered":"2024-04-25T16:28:20.715683+0000","last_clean":"2024-04-25T16:28:20.715683+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:28:20.715683+0000","last_undegraded":"2024-04-25T16:28:20.715683+0000","last_fullsized":"2024-04-25T16:28:20.715683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:20.715547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:20.715547+0000","last_clean_scrub_stamp":"2024-04-25T16:28:20.715547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:54.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:09.384028+0000","last_change":"2024-04-25T16:28:09.384108+0000","last_active":"2024-04-25T16:28:09.384028+0000","last_peered":"2024-04-25T16:28:09.384028+0000","last_clean":"2024-04-25T16:28:09.384028+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:28:09.384028+0000","last_undegraded":"2024-04-25T16:28:09.384028+0000","last_fullsized":"2024-04-25T16:28:09.384028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:09.383913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:09.383913+0000","last_clean_scrub_stamp":"2024-04-25T16:28:09.383913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:10.807069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.386653+0000","last_change":"2024-04-25T16:28:17.386677+0000","last_active":"2024-04-25T16:28:17.386653+0000","last_peered":"2024-04-25T16:28:17.386653+0000","last_clean":"2024-04-25T16:28:17.386653+0000","last_became_active":"2024-04-25T16:27:36.751463+0000","last_became_peered":"2024-04-25T16:27:36.751463+0000","last_unstale":"2024-04-25T16:28:17.386653+0000","last_undegraded":"2024-04-25T16:28:17.386653+0000","last_fullsized":"2024-04-25T16:28:17.386653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:17.386580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:17.386580+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.386580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:19.467904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:18.829186+0000","last_change":"2024-04-25T16:28:18.829238+0000","last_active":"2024-04-25T16:28:18.829186+0000","last_peered":"2024-04-25T16:28:18.829186+0000","last_clean":"2024-04-25T16:28:18.829186+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:28:18.829186+0000","last_undegraded":"2024-04-25T16:28:18.829186+0000","last_fullsized":"2024-04-25T16:28:18.829186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:18.829066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:18.829066+0000","last_clean_scrub_stamp":"2024-04-25T16:28:18.829066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:48.504622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697515+0000","last_change":"2024-04-25T16:26:55.561309+0000","last_active":"2024-04-25T16:27:36.697515+0000","last_peered":"2024-04-25T16:27:36.697515+0000","last_clean":"2024-04-25T16:27:36.697515+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:27:36.697515+0000","last_undegraded":"2024-04-25T16:27:36.697515+0000","last_fullsized":"2024-04-25T16:27:36.697515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:11.093016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:22.895366+0000","last_change":"2024-04-25T16:28:22.895399+0000","last_active":"2024-04-25T16:28:22.895366+0000","last_peered":"2024-04-25T16:28:22.895366+0000","last_clean":"2024-04-25T16:28:22.895366+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:28:22.895366+0000","last_undegraded":"2024-04-25T16:28:22.895366+0000","last_fullsized":"2024-04-25T16:28:22.895366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:22.895238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:22.895238+0000","last_clean_scrub_stamp":"2024-04-25T16:28:22.895238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:24.318394+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:04.584950+0000","last_change":"2024-04-25T16:28:04.584991+0000","last_active":"2024-04-25T16:28:04.584950+0000","last_peered":"2024-04-25T16:28:04.584950+0000","last_clean":"2024-04-25T16:28:04.584950+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:28:04.584950+0000","last_undegraded":"2024-04-25T16:28:04.584950+0000","last_fullsized":"2024-04-25T16:28:04.584950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:04.584863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:04.584863+0000","last_clean_scrub_stamp":"2024-04-25T16:28:04.584863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:52.800183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.700099+0000","last_change":"2024-04-25T16:26:55.557030+0000","last_active":"2024-04-25T16:27:36.700099+0000","last_peered":"2024-04-25T16:27:36.700099+0000","last_clean":"2024-04-25T16:27:36.700099+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:27:36.700099+0000","last_undegraded":"2024-04-25T16:27:36.700099+0000","last_fullsized":"2024-04-25T16:27:36.700099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750390+0000","last_change":"2024-04-25T16:27:00.603420+0000","last_active":"2024-04-25T16:27:36.750390+0000","last_peered":"2024-04-25T16:27:36.750390+0000","last_clean":"2024-04-25T16:27:36.750390+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:27:36.750390+0000","last_undegraded":"2024-04-25T16:27:36.750390+0000","last_fullsized":"2024-04-25T16:27:36.750390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:25.674010+0000","last_change":"2024-04-25T16:28:25.674052+0000","last_active":"2024-04-25T16:28:25.674010+0000","last_peered":"2024-04-25T16:28:25.674010+0000","last_clean":"2024-04-25T16:28:25.674010+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:28:25.674010+0000","last_undegraded":"2024-04-25T16:28:25.674010+0000","last_fullsized":"2024-04-25T16:28:25.674010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:25.673891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:25.673891+0000","last_clean_scrub_stamp":"2024-04-25T16:28:25.673891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:09.763480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:07.420778+0000","last_change":"2024-04-25T16:28:07.420852+0000","last_active":"2024-04-25T16:28:07.420778+0000","last_peered":"2024-04-25T16:28:07.420778+0000","last_clean":"2024-04-25T16:28:07.420778+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:28:07.420778+0000","last_undegraded":"2024-04-25T16:28:07.420778+0000","last_fullsized":"2024-04-25T16:28:07.420778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:07.420639+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:07.420639+0000","last_clean_scrub_stamp":"2024-04-25T16:28:07.420639+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:19.658768+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693793+0000","last_change":"2024-04-25T16:27:00.602776+0000","last_active":"2024-04-25T16:27:36.693793+0000","last_peered":"2024-04-25T16:27:36.693793+0000","last_clean":"2024-04-25T16:27:36.693793+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:27:36.693793+0000","last_undegraded":"2024-04-25T16:27:36.693793+0000","last_fullsized":"2024-04-25T16:27:36.693793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:11.308379+0000","last_change":"2024-04-25T16:28:11.308407+0000","last_active":"2024-04-25T16:28:11.308379+0000","last_peered":"2024-04-25T16:28:11.308379+0000","last_clean":"2024-04-25T16:28:11.308379+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:28:11.308379+0000","last_undegraded":"2024-04-25T16:28:11.308379+0000","last_fullsized":"2024-04-25T16:28:11.308379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:11.308277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:11.308277+0000","last_clean_scrub_stamp":"2024-04-25T16:28:11.308277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:41.361653+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698561+0000","last_change":"2024-04-25T16:26:55.559713+0000","last_active":"2024-04-25T16:27:36.698561+0000","last_peered":"2024-04-25T16:27:36.698561+0000","last_clean":"2024-04-25T16:27:36.698561+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:27:36.698561+0000","last_undegraded":"2024-04-25T16:27:36.698561+0000","last_fullsized":"2024-04-25T16:27:36.698561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.038473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.540825+0000","last_change":"2024-04-25T16:28:13.540871+0000","last_active":"2024-04-25T16:28:13.540825+0000","last_peered":"2024-04-25T16:28:13.540825+0000","last_clean":"2024-04-25T16:28:13.540825+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:28:13.540825+0000","last_undegraded":"2024-04-25T16:28:13.540825+0000","last_fullsized":"2024-04-25T16:28:13.540825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.540729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.540729+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.540729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:42.039219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764847+0000","last_change":"2024-04-25T16:26:56.564794+0000","last_active":"2024-04-25T16:27:36.764847+0000","last_peered":"2024-04-25T16:27:36.764847+0000","last_clean":"2024-04-25T16:27:36.764847+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:27:36.764847+0000","last_undegraded":"2024-04-25T16:27:36.764847+0000","last_fullsized":"2024-04-25T16:27:36.764847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:06.422639+0000","last_change":"2024-04-25T16:28:06.422702+0000","last_active":"2024-04-25T16:28:06.422639+0000","last_peered":"2024-04-25T16:28:06.422639+0000","last_clean":"2024-04-25T16:28:06.422639+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:28:06.422639+0000","last_undegraded":"2024-04-25T16:28:06.422639+0000","last_fullsized":"2024-04-25T16:28:06.422639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:06.422514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:06.422514+0000","last_clean_scrub_stamp":"2024-04-25T16:28:06.422514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:08.503838+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:18.386021+0000","last_change":"2024-04-25T16:28:18.386048+0000","last_active":"2024-04-25T16:28:18.386021+0000","last_peered":"2024-04-25T16:28:18.386021+0000","last_clean":"2024-04-25T16:28:18.386021+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:28:18.386021+0000","last_undegraded":"2024-04-25T16:28:18.386021+0000","last_fullsized":"2024-04-25T16:28:18.386021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:18.385937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:18.385937+0000","last_clean_scrub_stamp":"2024-04-25T16:28:18.385937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:46.884427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:20.901109+0000","last_change":"2024-04-25T16:28:20.901136+0000","last_active":"2024-04-25T16:28:20.901109+0000","last_peered":"2024-04-25T16:28:20.901109+0000","last_clean":"2024-04-25T16:28:20.901109+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:28:20.901109+0000","last_undegraded":"2024-04-25T16:28:20.901109+0000","last_fullsized":"2024-04-25T16:28:20.901109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:20.901004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:20.901004+0000","last_clean_scrub_stamp":"2024-04-25T16:28:20.901004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:33.139133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:25.066954+0000","last_change":"2024-04-25T16:28:25.066982+0000","last_active":"2024-04-25T16:28:25.066954+0000","last_peered":"2024-04-25T16:28:25.066954+0000","last_clean":"2024-04-25T16:28:25.066954+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:28:25.066954+0000","last_undegraded":"2024-04-25T16:28:25.066954+0000","last_fullsized":"2024-04-25T16:28:25.066954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:25.066856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:25.066856+0000","last_clean_scrub_stamp":"2024-04-25T16:28:25.066856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:55.453046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:16.382966+0000","last_change":"2024-04-25T16:28:16.382994+0000","last_active":"2024-04-25T16:28:16.382966+0000","last_peered":"2024-04-25T16:28:16.382966+0000","last_clean":"2024-04-25T16:28:16.382966+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:28:16.382966+0000","last_undegraded":"2024-04-25T16:28:16.382966+0000","last_fullsized":"2024-04-25T16:28:16.382966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:16.382863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:16.382863+0000","last_clean_scrub_stamp":"2024-04-25T16:28:16.382863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:05.906410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:10.338608+0000","last_change":"2024-04-25T16:28:10.338634+0000","last_active":"2024-04-25T16:28:10.338608+0000","last_peered":"2024-04-25T16:28:10.338608+0000","last_clean":"2024-04-25T16:28:10.338608+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:28:10.338608+0000","last_undegraded":"2024-04-25T16:28:10.338608+0000","last_fullsized":"2024-04-25T16:28:10.338608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:10.338507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:10.338507+0000","last_clean_scrub_stamp":"2024-04-25T16:28:10.338507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:43.391783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:10.476816+0000","last_change":"2024-04-25T16:28:10.476861+0000","last_active":"2024-04-25T16:28:10.476816+0000","last_peered":"2024-04-25T16:28:10.476816+0000","last_clean":"2024-04-25T16:28:10.476816+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:28:10.476816+0000","last_undegraded":"2024-04-25T16:28:10.476816+0000","last_fullsized":"2024-04-25T16:28:10.476816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:10.476675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:10.476675+0000","last_clean_scrub_stamp":"2024-04-25T16:28:10.476675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:18:52.884389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.339976+0000","last_change":"2024-04-25T16:28:17.340002+0000","last_active":"2024-04-25T16:28:17.339976+0000","last_peered":"2024-04-25T16:28:17.339976+0000","last_clean":"2024-04-25T16:28:17.339976+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:28:17.339976+0000","last_undegraded":"2024-04-25T16:28:17.339976+0000","last_fullsized":"2024-04-25T16:28:17.339976+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:17.339888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:17.339888+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.339888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:42.347432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:06.566517+0000","last_change":"2024-04-25T16:28:06.566566+0000","last_active":"2024-04-25T16:28:06.566517+0000","last_peered":"2024-04-25T16:28:06.566517+0000","last_clean":"2024-04-25T16:28:06.566517+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:28:06.566517+0000","last_undegraded":"2024-04-25T16:28:06.566517+0000","last_fullsized":"2024-04-25T16:28:06.566517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:06.566415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:06.566415+0000","last_clean_scrub_stamp":"2024-04-25T16:28:06.566415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:10.972339+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697357+0000","last_change":"2024-04-25T16:26:55.561669+0000","last_active":"2024-04-25T16:27:36.697357+0000","last_peered":"2024-04-25T16:27:36.697357+0000","last_clean":"2024-04-25T16:27:36.697357+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:27:36.697357+0000","last_undegraded":"2024-04-25T16:27:36.697357+0000","last_fullsized":"2024-04-25T16:27:36.697357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:35.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.545485+0000","last_change":"2024-04-25T16:28:14.545529+0000","last_active":"2024-04-25T16:28:14.545485+0000","last_peered":"2024-04-25T16:28:14.545485+0000","last_clean":"2024-04-25T16:28:14.545485+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:28:14.545485+0000","last_undegraded":"2024-04-25T16:28:14.545485+0000","last_fullsized":"2024-04-25T16:28:14.545485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.545387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.545387+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.545387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:37.777713+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.738461+0000","last_change":"2024-04-25T16:28:15.738503+0000","last_active":"2024-04-25T16:28:15.738461+0000","last_peered":"2024-04-25T16:28:15.738461+0000","last_clean":"2024-04-25T16:28:15.738461+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:28:15.738461+0000","last_undegraded":"2024-04-25T16:28:15.738461+0000","last_fullsized":"2024-04-25T16:28:15.738461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.738268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.738268+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.738268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:45.791644+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:24.089835+0000","last_change":"2024-04-25T16:28:24.089894+0000","last_active":"2024-04-25T16:28:24.089835+0000","last_peered":"2024-04-25T16:28:24.089835+0000","last_clean":"2024-04-25T16:28:24.089835+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:28:24.089835+0000","last_undegraded":"2024-04-25T16:28:24.089835+0000","last_fullsized":"2024-04-25T16:28:24.089835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:24.089728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:24.089728+0000","last_clean_scrub_stamp":"2024-04-25T16:28:24.089728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:12.068224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697495+0000","last_change":"2024-04-25T16:26:44.599634+0000","last_active":"2024-04-25T16:27:36.697495+0000","last_peered":"2024-04-25T16:27:36.697495+0000","last_clean":"2024-04-25T16:27:36.697495+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:27:36.697495+0000","last_undegraded":"2024-04-25T16:27:36.697495+0000","last_fullsized":"2024-04-25T16:27:36.697495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:43.492401+0000","last_clean_scrub_stamp":"2024-04-25T16:26:43.492401+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:50.102469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698869+0000","last_change":"2024-04-25T16:27:00.605134+0000","last_active":"2024-04-25T16:27:36.698869+0000","last_peered":"2024-04-25T16:27:36.698869+0000","last_clean":"2024-04-25T16:27:36.698869+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:27:36.698869+0000","last_undegraded":"2024-04-25T16:27:36.698869+0000","last_fullsized":"2024-04-25T16:27:36.698869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698193+0000","last_change":"2024-04-25T16:26:56.572195+0000","last_active":"2024-04-25T16:27:36.698193+0000","last_peered":"2024-04-25T16:27:36.698193+0000","last_clean":"2024-04-25T16:27:36.698193+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:27:36.698193+0000","last_undegraded":"2024-04-25T16:27:36.698193+0000","last_fullsized":"2024-04-25T16:27:36.698193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:22.610623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:11.500122+0000","last_change":"2024-04-25T16:28:11.500162+0000","last_active":"2024-04-25T16:28:11.500122+0000","last_peered":"2024-04-25T16:28:11.500122+0000","last_clean":"2024-04-25T16:28:11.500122+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:28:11.500122+0000","last_undegraded":"2024-04-25T16:28:11.500122+0000","last_fullsized":"2024-04-25T16:28:11.500122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:11.500002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:11.500002+0000","last_clean_scrub_stamp":"2024-04-25T16:28:11.500002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:42.358174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697341+0000","last_change":"2024-04-25T16:27:00.602367+0000","last_active":"2024-04-25T16:27:36.697341+0000","last_peered":"2024-04-25T16:27:36.697341+0000","last_clean":"2024-04-25T16:27:36.697341+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:27:36.697341+0000","last_undegraded":"2024-04-25T16:27:36.697341+0000","last_fullsized":"2024-04-25T16:27:36.697341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764626+0000","last_change":"2024-04-25T16:26:56.565836+0000","last_active":"2024-04-25T16:27:36.764626+0000","last_peered":"2024-04-25T16:27:36.764626+0000","last_clean":"2024-04-25T16:27:36.764626+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:27:36.764626+0000","last_undegraded":"2024-04-25T16:27:36.764626+0000","last_fullsized":"2024-04-25T16:27:36.764626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:05.433386+0000","last_change":"2024-04-25T16:28:05.433416+0000","last_active":"2024-04-25T16:28:05.433386+0000","last_peered":"2024-04-25T16:28:05.433386+0000","last_clean":"2024-04-25T16:28:05.433386+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:28:05.433386+0000","last_undegraded":"2024-04-25T16:28:05.433386+0000","last_fullsized":"2024-04-25T16:28:05.433386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-25T16:28:05.433298+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-25T16:28:05.433298+0000","last_clean_scrub_stamp":"2024-04-25T16:28:05.433298+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:35.108855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:11.460612+0000","last_change":"2024-04-25T16:28:11.460659+0000","last_active":"2024-04-25T16:28:11.460612+0000","last_peered":"2024-04-25T16:28:11.460612+0000","last_clean":"2024-04-25T16:28:11.460612+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:28:11.460612+0000","last_undegraded":"2024-04-25T16:28:11.460612+0000","last_fullsized":"2024-04-25T16:28:11.460612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:11.460488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:11.460488+0000","last_clean_scrub_stamp":"2024-04-25T16:28:11.460488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:00.984034+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699318+0000","last_change":"2024-04-25T16:26:56.569161+0000","last_active":"2024-04-25T16:27:36.699318+0000","last_peered":"2024-04-25T16:27:36.699318+0000","last_clean":"2024-04-25T16:27:36.699318+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:27:36.699318+0000","last_undegraded":"2024-04-25T16:27:36.699318+0000","last_fullsized":"2024-04-25T16:27:36.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:59.500236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:08.554492+0000","last_change":"2024-04-25T16:28:08.554528+0000","last_active":"2024-04-25T16:28:08.554492+0000","last_peered":"2024-04-25T16:28:08.554492+0000","last_clean":"2024-04-25T16:28:08.554492+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:28:08.554492+0000","last_undegraded":"2024-04-25T16:28:08.554492+0000","last_fullsized":"2024-04-25T16:28:08.554492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:08.554406+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:08.554406+0000","last_clean_scrub_stamp":"2024-04-25T16:28:08.554406+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:53.392341+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.763929+0000","last_change":"2024-04-25T16:27:00.605249+0000","last_active":"2024-04-25T16:27:36.763929+0000","last_peered":"2024-04-25T16:27:36.763929+0000","last_clean":"2024-04-25T16:27:36.763929+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:27:36.763929+0000","last_undegraded":"2024-04-25T16:27:36.763929+0000","last_fullsized":"2024-04-25T16:27:36.763929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698846+0000","last_change":"2024-04-25T16:26:56.569216+0000","last_active":"2024-04-25T16:27:36.698846+0000","last_peered":"2024-04-25T16:27:36.698846+0000","last_clean":"2024-04-25T16:27:36.698846+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:27:36.698846+0000","last_undegraded":"2024-04-25T16:27:36.698846+0000","last_fullsized":"2024-04-25T16:27:36.698846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:45.366261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.808515+0000","last_change":"2024-04-25T16:28:17.808562+0000","last_active":"2024-04-25T16:28:17.808515+0000","last_peered":"2024-04-25T16:28:17.808515+0000","last_clean":"2024-04-25T16:28:17.808515+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:28:17.808515+0000","last_undegraded":"2024-04-25T16:28:17.808515+0000","last_fullsized":"2024-04-25T16:28:17.808515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-25T16:28:17.808422+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-25T16:28:17.808422+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.808422+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:25.703132+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.750222+0000","last_change":"2024-04-25T16:27:00.589765+0000","last_active":"2024-04-25T16:27:36.750222+0000","last_peered":"2024-04-25T16:27:36.750222+0000","last_clean":"2024-04-25T16:27:36.750222+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:27:36.750222+0000","last_undegraded":"2024-04-25T16:27:36.750222+0000","last_fullsized":"2024-04-25T16:27:36.750222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698175+0000","last_change":"2024-04-25T16:26:56.569453+0000","last_active":"2024-04-25T16:27:36.698175+0000","last_peered":"2024-04-25T16:27:36.698175+0000","last_clean":"2024-04-25T16:27:36.698175+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:27:36.698175+0000","last_undegraded":"2024-04-25T16:27:36.698175+0000","last_fullsized":"2024-04-25T16:27:36.698175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:33.853305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'12","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:09.521548+0000","last_change":"2024-04-25T16:28:09.521591+0000","last_active":"2024-04-25T16:28:09.521548+0000","last_peered":"2024-04-25T16:28:09.521548+0000","last_clean":"2024-04-25T16:28:09.521548+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:28:09.521548+0000","last_undegraded":"2024-04-25T16:28:09.521548+0000","last_fullsized":"2024-04-25T16:28:09.521548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-25T16:28:09.521455+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-25T16:28:09.521455+0000","last_clean_scrub_stamp":"2024-04-25T16:28:09.521455+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:31.495884+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.493514+0000","last_change":"2024-04-25T16:28:12.493559+0000","last_active":"2024-04-25T16:28:12.493514+0000","last_peered":"2024-04-25T16:28:12.493514+0000","last_clean":"2024-04-25T16:28:12.493514+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:28:12.493514+0000","last_undegraded":"2024-04-25T16:28:12.493514+0000","last_fullsized":"2024-04-25T16:28:12.493514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.493394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.493394+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.493394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:37.500938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697586+0000","last_change":"2024-04-25T16:26:56.555642+0000","last_active":"2024-04-25T16:27:36.697586+0000","last_peered":"2024-04-25T16:27:36.697586+0000","last_clean":"2024-04-25T16:27:36.697586+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:27:36.697586+0000","last_undegraded":"2024-04-25T16:27:36.697586+0000","last_fullsized":"2024-04-25T16:27:36.697586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:28.144139+0000","last_change":"2024-04-25T16:28:28.144172+0000","last_active":"2024-04-25T16:28:28.144139+0000","last_peered":"2024-04-25T16:28:28.144139+0000","last_clean":"2024-04-25T16:28:28.144139+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:28:28.144139+0000","last_undegraded":"2024-04-25T16:28:28.144139+0000","last_fullsized":"2024-04-25T16:28:28.144139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:28.144045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:28.144045+0000","last_clean_scrub_stamp":"2024-04-25T16:28:28.144045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.339354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694987+0000","last_change":"2024-04-25T16:27:00.605649+0000","last_active":"2024-04-25T16:27:36.694987+0000","last_peered":"2024-04-25T16:27:36.694987+0000","last_clean":"2024-04-25T16:27:36.694987+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:27:36.694987+0000","last_undegraded":"2024-04-25T16:27:36.694987+0000","last_fullsized":"2024-04-25T16:27:36.694987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698423+0000","last_change":"2024-04-25T16:26:56.570722+0000","last_active":"2024-04-25T16:27:36.698423+0000","last_peered":"2024-04-25T16:27:36.698423+0000","last_clean":"2024-04-25T16:27:36.698423+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:27:36.698423+0000","last_undegraded":"2024-04-25T16:27:36.698423+0000","last_fullsized":"2024-04-25T16:27:36.698423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:56:32.793574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.989783+0000","last_change":"2024-04-25T16:28:26.989830+0000","last_active":"2024-04-25T16:28:26.989783+0000","last_peered":"2024-04-25T16:28:26.989783+0000","last_clean":"2024-04-25T16:28:26.989783+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:28:26.989783+0000","last_undegraded":"2024-04-25T16:28:26.989783+0000","last_fullsized":"2024-04-25T16:28:26.989783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.989652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.989652+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.989652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:59.374695+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.694334+0000","last_change":"2024-04-25T16:27:00.604031+0000","last_active":"2024-04-25T16:27:36.694334+0000","last_peered":"2024-04-25T16:27:36.694334+0000","last_clean":"2024-04-25T16:27:36.694334+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:27:36.694334+0000","last_undegraded":"2024-04-25T16:27:36.694334+0000","last_fullsized":"2024-04-25T16:27:36.694334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:08.527534+0000","last_change":"2024-04-25T16:28:08.527555+0000","last_active":"2024-04-25T16:28:08.527534+0000","last_peered":"2024-04-25T16:28:08.527534+0000","last_clean":"2024-04-25T16:28:08.527534+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:28:08.527534+0000","last_undegraded":"2024-04-25T16:28:08.527534+0000","last_fullsized":"2024-04-25T16:28:08.527534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:08.527460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:08.527460+0000","last_clean_scrub_stamp":"2024-04-25T16:28:08.527460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:46.063221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'4","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:00.481192+0000","last_change":"2024-04-25T16:28:00.481248+0000","last_active":"2024-04-25T16:28:00.481192+0000","last_peered":"2024-04-25T16:28:00.481192+0000","last_clean":"2024-04-25T16:28:00.481192+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:28:00.481192+0000","last_undegraded":"2024-04-25T16:28:00.481192+0000","last_fullsized":"2024-04-25T16:28:00.481192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-25T16:28:00.481039+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-25T16:28:00.481039+0000","last_clean_scrub_stamp":"2024-04-25T16:28:00.481039+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:08.375749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764565+0000","last_change":"2024-04-25T16:27:00.605128+0000","last_active":"2024-04-25T16:27:36.764565+0000","last_peered":"2024-04-25T16:27:36.764565+0000","last_clean":"2024-04-25T16:27:36.764565+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:27:36.764565+0000","last_undegraded":"2024-04-25T16:27:36.764565+0000","last_fullsized":"2024-04-25T16:27:36.764565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698073+0000","last_change":"2024-04-25T16:26:56.568714+0000","last_active":"2024-04-25T16:27:36.698073+0000","last_peered":"2024-04-25T16:27:36.698073+0000","last_clean":"2024-04-25T16:27:36.698073+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:27:36.698073+0000","last_undegraded":"2024-04-25T16:27:36.698073+0000","last_fullsized":"2024-04-25T16:27:36.698073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.069111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'28","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.134875+0000","last_change":"2024-04-25T16:28:27.134922+0000","last_active":"2024-04-25T16:28:27.134875+0000","last_peered":"2024-04-25T16:28:27.134875+0000","last_clean":"2024-04-25T16:28:27.134875+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:28:27.134875+0000","last_undegraded":"2024-04-25T16:28:27.134875+0000","last_fullsized":"2024-04-25T16:28:27.134875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-25T16:28:27.134775+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-25T16:28:27.134775+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.134775+0000","objects_scrubbed":4,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:39.372904+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.359240+0000","last_change":"2024-04-25T16:28:15.359298+0000","last_active":"2024-04-25T16:28:15.359240+0000","last_peered":"2024-04-25T16:28:15.359240+0000","last_clean":"2024-04-25T16:28:15.359240+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:28:15.359240+0000","last_undegraded":"2024-04-25T16:28:15.359240+0000","last_fullsized":"2024-04-25T16:28:15.359240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.359100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.359100+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.359100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:27.480044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.458383+0000","last_change":"2024-04-25T16:28:15.458425+0000","last_active":"2024-04-25T16:28:15.458383+0000","last_peered":"2024-04-25T16:28:15.458383+0000","last_clean":"2024-04-25T16:28:15.458383+0000","last_became_active":"2024-04-25T16:27:36.700105+0000","last_became_peered":"2024-04-25T16:27:36.700105+0000","last_unstale":"2024-04-25T16:28:15.458383+0000","last_undegraded":"2024-04-25T16:28:15.458383+0000","last_fullsized":"2024-04-25T16:28:15.458383+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.458267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.458267+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.458267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:33.595091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:08.430920+0000","last_change":"2024-04-25T16:28:08.430951+0000","last_active":"2024-04-25T16:28:08.430920+0000","last_peered":"2024-04-25T16:28:08.430920+0000","last_clean":"2024-04-25T16:28:08.430920+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:28:08.430920+0000","last_undegraded":"2024-04-25T16:28:08.430920+0000","last_fullsized":"2024-04-25T16:28:08.430920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-25T16:28:08.430845+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-25T16:28:08.430845+0000","last_clean_scrub_stamp":"2024-04-25T16:28:08.430845+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:10.304775+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.773109+0000","last_change":"2024-04-25T16:28:27.773162+0000","last_active":"2024-04-25T16:28:27.773109+0000","last_peered":"2024-04-25T16:28:27.773109+0000","last_clean":"2024-04-25T16:28:27.773109+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:28:27.773109+0000","last_undegraded":"2024-04-25T16:28:27.773109+0000","last_fullsized":"2024-04-25T16:28:27.773109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:27.772993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:27.772993+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.772993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:00.158036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:09.489587+0000","last_change":"2024-04-25T16:28:09.489610+0000","last_active":"2024-04-25T16:28:09.489587+0000","last_peered":"2024-04-25T16:28:09.489587+0000","last_clean":"2024-04-25T16:28:09.489587+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:28:09.489587+0000","last_undegraded":"2024-04-25T16:28:09.489587+0000","last_fullsized":"2024-04-25T16:28:09.489587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:09.489480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:09.489480+0000","last_clean_scrub_stamp":"2024-04-25T16:28:09.489480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:41.874523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.969428+0000","last_change":"2024-04-25T16:28:27.969475+0000","last_active":"2024-04-25T16:28:27.969428+0000","last_peered":"2024-04-25T16:28:27.969428+0000","last_clean":"2024-04-25T16:28:27.969428+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:28:27.969428+0000","last_undegraded":"2024-04-25T16:28:27.969428+0000","last_fullsized":"2024-04-25T16:28:27.969428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:27.969287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:27.969287+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.969287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:40.090231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:43.312566+0000","last_change":"2024-04-25T16:27:00.585054+0000","last_active":"2024-04-25T16:27:43.312566+0000","last_peered":"2024-04-25T16:27:43.312566+0000","last_clean":"2024-04-25T16:27:43.312566+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:27:43.312566+0000","last_undegraded":"2024-04-25T16:27:43.312566+0000","last_fullsized":"2024-04-25T16:27:43.312566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764730+0000","last_change":"2024-04-25T16:26:56.572377+0000","last_active":"2024-04-25T16:27:36.764730+0000","last_peered":"2024-04-25T16:27:36.764730+0000","last_clean":"2024-04-25T16:27:36.764730+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:27:36.764730+0000","last_undegraded":"2024-04-25T16:27:36.764730+0000","last_fullsized":"2024-04-25T16:27:36.764730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.700008+0000","last_change":"2024-04-25T16:26:55.559580+0000","last_active":"2024-04-25T16:27:36.700008+0000","last_peered":"2024-04-25T16:27:36.700008+0000","last_clean":"2024-04-25T16:27:36.700008+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:27:36.700008+0000","last_undegraded":"2024-04-25T16:27:36.700008+0000","last_fullsized":"2024-04-25T16:27:36.700008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:54.523761+0000","last_clean_scrub_stamp":"2024-04-25T16:26:54.523761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.693917+0000","last_change":"2024-04-25T16:27:00.608026+0000","last_active":"2024-04-25T16:27:36.693917+0000","last_peered":"2024-04-25T16:27:36.693917+0000","last_clean":"2024-04-25T16:27:36.693917+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:27:36.693917+0000","last_undegraded":"2024-04-25T16:27:36.693917+0000","last_fullsized":"2024-04-25T16:27:36.693917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.532576+0000","last_change":"2024-04-25T16:28:15.532607+0000","last_active":"2024-04-25T16:28:15.532576+0000","last_peered":"2024-04-25T16:28:15.532576+0000","last_clean":"2024-04-25T16:28:15.532576+0000","last_became_active":"2024-04-25T16:27:36.702077+0000","last_became_peered":"2024-04-25T16:27:36.702077+0000","last_unstale":"2024-04-25T16:28:15.532576+0000","last_undegraded":"2024-04-25T16:28:15.532576+0000","last_fullsized":"2024-04-25T16:28:15.532576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.532484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.532484+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.532484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:26.513787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,2],"acting":[0,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:19.411015+0000","last_change":"2024-04-25T16:28:19.411051+0000","last_active":"2024-04-25T16:28:19.411015+0000","last_peered":"2024-04-25T16:28:19.411015+0000","last_clean":"2024-04-25T16:28:19.411015+0000","last_became_active":"2024-04-25T16:27:36.701427+0000","last_became_peered":"2024-04-25T16:27:36.701427+0000","last_unstale":"2024-04-25T16:28:19.411015+0000","last_undegraded":"2024-04-25T16:28:19.411015+0000","last_fullsized":"2024-04-25T16:28:19.411015+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:19.410911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:19.410911+0000","last_clean_scrub_stamp":"2024-04-25T16:28:19.410911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:42.643237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.018943+0000","last_change":"2024-04-25T16:28:26.018974+0000","last_active":"2024-04-25T16:28:26.018943+0000","last_peered":"2024-04-25T16:28:26.018943+0000","last_clean":"2024-04-25T16:28:26.018943+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:28:26.018943+0000","last_undegraded":"2024-04-25T16:28:26.018943+0000","last_fullsized":"2024-04-25T16:28:26.018943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.018830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.018830+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.018830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:01.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.766803+0000","last_change":"2024-04-25T16:27:36.766198+0000","last_active":"2024-04-25T16:27:36.766803+0000","last_peered":"2024-04-25T16:27:36.766803+0000","last_clean":"2024-04-25T16:27:36.766803+0000","last_became_active":"2024-04-25T16:27:36.765669+0000","last_became_peered":"2024-04-25T16:27:36.765669+0000","last_unstale":"2024-04-25T16:27:36.766803+0000","last_undegraded":"2024-04-25T16:27:36.766803+0000","last_fullsized":"2024-04-25T16:27:36.766803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:59.558966+0000","last_clean_scrub_stamp":"2024-04-25T16:26:59.558966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:21.676037+0000","last_change":"2024-04-25T16:28:21.676087+0000","last_active":"2024-04-25T16:28:21.676037+0000","last_peered":"2024-04-25T16:28:21.676037+0000","last_clean":"2024-04-25T16:28:21.676037+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:28:21.676037+0000","last_undegraded":"2024-04-25T16:28:21.676037+0000","last_fullsized":"2024-04-25T16:28:21.676037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:21.675943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:21.675943+0000","last_clean_scrub_stamp":"2024-04-25T16:28:21.675943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:46.683488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.750756+0000","last_change":"2024-04-25T16:28:17.750781+0000","last_active":"2024-04-25T16:28:17.750756+0000","last_peered":"2024-04-25T16:28:17.750756+0000","last_clean":"2024-04-25T16:28:17.750756+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:28:17.750756+0000","last_undegraded":"2024-04-25T16:28:17.750756+0000","last_fullsized":"2024-04-25T16:28:17.750756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:17.750678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:17.750678+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.750678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.286439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.158150+0000","last_change":"2024-04-25T16:28:26.158182+0000","last_active":"2024-04-25T16:28:26.158150+0000","last_peered":"2024-04-25T16:28:26.158150+0000","last_clean":"2024-04-25T16:28:26.158150+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:28:26.158150+0000","last_undegraded":"2024-04-25T16:28:26.158150+0000","last_fullsized":"2024-04-25T16:28:26.158150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T16:28:26.158055+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T16:28:26.158055+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.158055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:28.239379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:07.527654+0000","last_change":"2024-04-25T16:28:07.527683+0000","last_active":"2024-04-25T16:28:07.527654+0000","last_peered":"2024-04-25T16:28:07.527654+0000","last_clean":"2024-04-25T16:28:07.527654+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:28:07.527654+0000","last_undegraded":"2024-04-25T16:28:07.527654+0000","last_fullsized":"2024-04-25T16:28:07.527654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:07.527553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:07.527553+0000","last_clean_scrub_stamp":"2024-04-25T16:28:07.527553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:26.746231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.456014+0000","last_change":"2024-04-25T16:28:14.456042+0000","last_active":"2024-04-25T16:28:14.456014+0000","last_peered":"2024-04-25T16:28:14.456014+0000","last_clean":"2024-04-25T16:28:14.456014+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:28:14.456014+0000","last_undegraded":"2024-04-25T16:28:14.456014+0000","last_fullsized":"2024-04-25T16:28:14.456014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.455918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.455918+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.455918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:32.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:19.719920+0000","last_change":"2024-04-25T16:28:19.719967+0000","last_active":"2024-04-25T16:28:19.719920+0000","last_peered":"2024-04-25T16:28:19.719920+0000","last_clean":"2024-04-25T16:28:19.719920+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:28:19.719920+0000","last_undegraded":"2024-04-25T16:28:19.719920+0000","last_fullsized":"2024-04-25T16:28:19.719920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T16:28:19.719762+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T16:28:19.719762+0000","last_clean_scrub_stamp":"2024-04-25T16:28:19.719762+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:34.965212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:19.861440+0000","last_change":"2024-04-25T16:28:19.861473+0000","last_active":"2024-04-25T16:28:19.861440+0000","last_peered":"2024-04-25T16:28:19.861440+0000","last_clean":"2024-04-25T16:28:19.861440+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:28:19.861440+0000","last_undegraded":"2024-04-25T16:28:19.861440+0000","last_fullsized":"2024-04-25T16:28:19.861440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:19.861334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:19.861334+0000","last_clean_scrub_stamp":"2024-04-25T16:28:19.861334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:21.942657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.689327+0000","last_change":"2024-04-25T16:28:26.689356+0000","last_active":"2024-04-25T16:28:26.689327+0000","last_peered":"2024-04-25T16:28:26.689327+0000","last_clean":"2024-04-25T16:28:26.689327+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:28:26.689327+0000","last_undegraded":"2024-04-25T16:28:26.689327+0000","last_fullsized":"2024-04-25T16:28:26.689327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.689225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.689225+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.689225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:04.033610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697459+0000","last_change":"2024-04-25T16:26:56.555588+0000","last_active":"2024-04-25T16:27:36.697459+0000","last_peered":"2024-04-25T16:27:36.697459+0000","last_clean":"2024-04-25T16:27:36.697459+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:27:36.697459+0000","last_undegraded":"2024-04-25T16:27:36.697459+0000","last_fullsized":"2024-04-25T16:27:36.697459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.449230+0000","last_change":"2024-04-25T16:28:13.449257+0000","last_active":"2024-04-25T16:28:13.449230+0000","last_peered":"2024-04-25T16:28:13.449230+0000","last_clean":"2024-04-25T16:28:13.449230+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:28:13.449230+0000","last_undegraded":"2024-04-25T16:28:13.449230+0000","last_fullsized":"2024-04-25T16:28:13.449230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.449148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.449148+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.449148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:13.025915+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:16.427112+0000","last_change":"2024-04-25T16:28:16.427156+0000","last_active":"2024-04-25T16:28:16.427112+0000","last_peered":"2024-04-25T16:28:16.427112+0000","last_clean":"2024-04-25T16:28:16.427112+0000","last_became_active":"2024-04-25T16:27:36.701156+0000","last_became_peered":"2024-04-25T16:27:36.701156+0000","last_unstale":"2024-04-25T16:28:16.427112+0000","last_undegraded":"2024-04-25T16:28:16.427112+0000","last_fullsized":"2024-04-25T16:28:16.427112+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:16.426964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:16.426964+0000","last_clean_scrub_stamp":"2024-04-25T16:28:16.426964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:19.917910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:05.547570+0000","last_change":"2024-04-25T16:28:05.547614+0000","last_active":"2024-04-25T16:28:05.547570+0000","last_peered":"2024-04-25T16:28:05.547570+0000","last_clean":"2024-04-25T16:28:05.547570+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:28:05.547570+0000","last_undegraded":"2024-04-25T16:28:05.547570+0000","last_fullsized":"2024-04-25T16:28:05.547570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:05.547456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:05.547456+0000","last_clean_scrub_stamp":"2024-04-25T16:28:05.547456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:38.600732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:24.668238+0000","last_change":"2024-04-25T16:28:24.668282+0000","last_active":"2024-04-25T16:28:24.668238+0000","last_peered":"2024-04-25T16:28:24.668238+0000","last_clean":"2024-04-25T16:28:24.668238+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:28:24.668238+0000","last_undegraded":"2024-04-25T16:28:24.668238+0000","last_fullsized":"2024-04-25T16:28:24.668238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:24.668106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:24.668106+0000","last_clean_scrub_stamp":"2024-04-25T16:28:24.668106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.247900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.301436+0000","last_change":"2024-04-25T16:28:13.301477+0000","last_active":"2024-04-25T16:28:13.301436+0000","last_peered":"2024-04-25T16:28:13.301436+0000","last_clean":"2024-04-25T16:28:13.301436+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:28:13.301436+0000","last_undegraded":"2024-04-25T16:28:13.301436+0000","last_fullsized":"2024-04-25T16:28:13.301436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.301298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.301298+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.301298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:50.837059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699583+0000","last_change":"2024-04-25T16:26:56.557918+0000","last_active":"2024-04-25T16:27:36.699583+0000","last_peered":"2024-04-25T16:27:36.699583+0000","last_clean":"2024-04-25T16:27:36.699583+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:27:36.699583+0000","last_undegraded":"2024-04-25T16:27:36.699583+0000","last_fullsized":"2024-04-25T16:27:36.699583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:22.696689+0000","last_change":"2024-04-25T16:28:22.696720+0000","last_active":"2024-04-25T16:28:22.696689+0000","last_peered":"2024-04-25T16:28:22.696689+0000","last_clean":"2024-04-25T16:28:22.696689+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:28:22.696689+0000","last_undegraded":"2024-04-25T16:28:22.696689+0000","last_fullsized":"2024-04-25T16:28:22.696689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:22.696584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:22.696584+0000","last_clean_scrub_stamp":"2024-04-25T16:28:22.696584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.771045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764075+0000","last_change":"2024-04-25T16:26:56.564377+0000","last_active":"2024-04-25T16:27:36.764075+0000","last_peered":"2024-04-25T16:27:36.764075+0000","last_clean":"2024-04-25T16:27:36.764075+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:27:36.764075+0000","last_undegraded":"2024-04-25T16:27:36.764075+0000","last_fullsized":"2024-04-25T16:27:36.764075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:16.727278+0000","last_change":"2024-04-25T16:28:16.727319+0000","last_active":"2024-04-25T16:28:16.727278+0000","last_peered":"2024-04-25T16:28:16.727278+0000","last_clean":"2024-04-25T16:28:16.727278+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:28:16.727278+0000","last_undegraded":"2024-04-25T16:28:16.727278+0000","last_fullsized":"2024-04-25T16:28:16.727278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:16.727153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:16.727153+0000","last_clean_scrub_stamp":"2024-04-25T16:28:16.727153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:35.945831+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764747+0000","last_change":"2024-04-25T16:26:56.572515+0000","last_active":"2024-04-25T16:27:36.764747+0000","last_peered":"2024-04-25T16:27:36.764747+0000","last_clean":"2024-04-25T16:27:36.764747+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:27:36.764747+0000","last_undegraded":"2024-04-25T16:27:36.764747+0000","last_fullsized":"2024-04-25T16:27:36.764747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:10.471272+0000","last_change":"2024-04-25T16:28:10.471321+0000","last_active":"2024-04-25T16:28:10.471272+0000","last_peered":"2024-04-25T16:28:10.471272+0000","last_clean":"2024-04-25T16:28:10.471272+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:28:10.471272+0000","last_undegraded":"2024-04-25T16:28:10.471272+0000","last_fullsized":"2024-04-25T16:28:10.471272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:10.471146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:10.471146+0000","last_clean_scrub_stamp":"2024-04-25T16:28:10.471146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:22.592090+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698827+0000","last_change":"2024-04-25T16:26:56.555817+0000","last_active":"2024-04-25T16:27:36.698827+0000","last_peered":"2024-04-25T16:27:36.698827+0000","last_clean":"2024-04-25T16:27:36.698827+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:27:36.698827+0000","last_undegraded":"2024-04-25T16:27:36.698827+0000","last_fullsized":"2024-04-25T16:27:36.698827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698354+0000","last_change":"2024-04-25T16:26:56.570824+0000","last_active":"2024-04-25T16:27:36.698354+0000","last_peered":"2024-04-25T16:27:36.698354+0000","last_clean":"2024-04-25T16:27:36.698354+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:27:36.698354+0000","last_undegraded":"2024-04-25T16:27:36.698354+0000","last_fullsized":"2024-04-25T16:27:36.698354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:06.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.493300+0000","last_change":"2024-04-25T16:28:12.493341+0000","last_active":"2024-04-25T16:28:12.493300+0000","last_peered":"2024-04-25T16:28:12.493300+0000","last_clean":"2024-04-25T16:28:12.493300+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:28:12.493300+0000","last_undegraded":"2024-04-25T16:28:12.493300+0000","last_fullsized":"2024-04-25T16:28:12.493300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.493175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.493175+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.493175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:08.943179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764935+0000","last_change":"2024-04-25T16:26:56.573193+0000","last_active":"2024-04-25T16:27:36.764935+0000","last_peered":"2024-04-25T16:27:36.764935+0000","last_clean":"2024-04-25T16:27:36.764935+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:27:36.764935+0000","last_undegraded":"2024-04-25T16:27:36.764935+0000","last_fullsized":"2024-04-25T16:27:36.764935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764090+0000","last_change":"2024-04-25T16:26:56.567012+0000","last_active":"2024-04-25T16:27:36.764090+0000","last_peered":"2024-04-25T16:27:36.764090+0000","last_clean":"2024-04-25T16:27:36.764090+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:27:36.764090+0000","last_undegraded":"2024-04-25T16:27:36.764090+0000","last_fullsized":"2024-04-25T16:27:36.764090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.339993+0000","last_change":"2024-04-25T16:28:14.340050+0000","last_active":"2024-04-25T16:28:14.339993+0000","last_peered":"2024-04-25T16:28:14.339993+0000","last_clean":"2024-04-25T16:28:14.339993+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:28:14.339993+0000","last_undegraded":"2024-04-25T16:28:14.339993+0000","last_fullsized":"2024-04-25T16:28:14.339993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.339872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.339872+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.339872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:46.724915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:03.574699+0000","last_change":"2024-04-25T16:28:03.574722+0000","last_active":"2024-04-25T16:28:03.574699+0000","last_peered":"2024-04-25T16:28:03.574699+0000","last_clean":"2024-04-25T16:28:03.574699+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:28:03.574699+0000","last_undegraded":"2024-04-25T16:28:03.574699+0000","last_fullsized":"2024-04-25T16:28:03.574699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:03.574620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:03.574620+0000","last_clean_scrub_stamp":"2024-04-25T16:28:03.574620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:30.769929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.723433+0000","last_change":"2024-04-25T16:28:14.723461+0000","last_active":"2024-04-25T16:28:14.723433+0000","last_peered":"2024-04-25T16:28:14.723433+0000","last_clean":"2024-04-25T16:28:14.723433+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:28:14.723433+0000","last_undegraded":"2024-04-25T16:28:14.723433+0000","last_fullsized":"2024-04-25T16:28:14.723433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.723299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.723299+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.723299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:47.776575+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:21.929950+0000","last_change":"2024-04-25T16:28:21.929981+0000","last_active":"2024-04-25T16:28:21.929950+0000","last_peered":"2024-04-25T16:28:21.929950+0000","last_clean":"2024-04-25T16:28:21.929950+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:28:21.929950+0000","last_undegraded":"2024-04-25T16:28:21.929950+0000","last_fullsized":"2024-04-25T16:28:21.929950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:21.929827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:21.929827+0000","last_clean_scrub_stamp":"2024-04-25T16:28:21.929827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:23.803757+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699126+0000","last_change":"2024-04-25T16:26:56.567580+0000","last_active":"2024-04-25T16:27:36.699126+0000","last_peered":"2024-04-25T16:27:36.699126+0000","last_clean":"2024-04-25T16:27:36.699126+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:27:36.699126+0000","last_undegraded":"2024-04-25T16:27:36.699126+0000","last_fullsized":"2024-04-25T16:27:36.699126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.699447+0000","last_change":"2024-04-25T16:26:56.563601+0000","last_active":"2024-04-25T16:27:36.699447+0000","last_peered":"2024-04-25T16:27:36.699447+0000","last_clean":"2024-04-25T16:27:36.699447+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:27:36.699447+0000","last_undegraded":"2024-04-25T16:27:36.699447+0000","last_fullsized":"2024-04-25T16:27:36.699447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.703786+0000","last_change":"2024-04-25T16:27:36.703786+0000","last_active":"2024-04-25T16:27:36.703786+0000","last_peered":"2024-04-25T16:27:36.703786+0000","last_clean":"2024-04-25T16:27:36.703786+0000","last_became_active":"2024-04-25T16:27:36.703485+0000","last_became_peered":"2024-04-25T16:27:36.703485+0000","last_unstale":"2024-04-25T16:27:36.703786+0000","last_undegraded":"2024-04-25T16:27:36.703786+0000","last_fullsized":"2024-04-25T16:27:36.703786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:51.980383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.704765+0000","last_change":"2024-04-25T16:28:13.704807+0000","last_active":"2024-04-25T16:28:13.704765+0000","last_peered":"2024-04-25T16:28:13.704765+0000","last_clean":"2024-04-25T16:28:13.704765+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:28:13.704765+0000","last_undegraded":"2024-04-25T16:28:13.704765+0000","last_fullsized":"2024-04-25T16:28:13.704765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.704644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.704644+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.704644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:15.127800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697810+0000","last_change":"2024-04-25T16:26:56.562906+0000","last_active":"2024-04-25T16:27:36.697810+0000","last_peered":"2024-04-25T16:27:36.697810+0000","last_clean":"2024-04-25T16:27:36.697810+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:27:36.697810+0000","last_undegraded":"2024-04-25T16:27:36.697810+0000","last_fullsized":"2024-04-25T16:27:36.697810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:00.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.764339+0000","last_change":"2024-04-25T16:26:56.566978+0000","last_active":"2024-04-25T16:27:36.764339+0000","last_peered":"2024-04-25T16:27:36.764339+0000","last_clean":"2024-04-25T16:27:36.764339+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:27:36.764339+0000","last_undegraded":"2024-04-25T16:27:36.764339+0000","last_fullsized":"2024-04-25T16:27:36.764339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.697835+0000","last_change":"2024-04-25T16:26:56.567641+0000","last_active":"2024-04-25T16:27:36.697835+0000","last_peered":"2024-04-25T16:27:36.697835+0000","last_clean":"2024-04-25T16:27:36.697835+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:27:36.697835+0000","last_undegraded":"2024-04-25T16:27:36.697835+0000","last_fullsized":"2024-04-25T16:27:36.697835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.389086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:22.083000+0000","last_change":"2024-04-25T16:28:22.083029+0000","last_active":"2024-04-25T16:28:22.083000+0000","last_peered":"2024-04-25T16:28:22.083000+0000","last_clean":"2024-04-25T16:28:22.083000+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:28:22.083000+0000","last_undegraded":"2024-04-25T16:28:22.083000+0000","last_fullsized":"2024-04-25T16:28:22.083000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:22.082903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:22.082903+0000","last_clean_scrub_stamp":"2024-04-25T16:28:22.082903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:49.278211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:02.525404+0000","last_change":"2024-04-25T16:28:02.525439+0000","last_active":"2024-04-25T16:28:02.525404+0000","last_peered":"2024-04-25T16:28:02.525404+0000","last_clean":"2024-04-25T16:28:02.525404+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:28:02.525404+0000","last_undegraded":"2024-04-25T16:28:02.525404+0000","last_fullsized":"2024-04-25T16:28:02.525404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:02.525282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:02.525282+0000","last_clean_scrub_stamp":"2024-04-25T16:28:02.525282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:14.763411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.309906+0000","last_change":"2024-04-25T16:28:12.309946+0000","last_active":"2024-04-25T16:28:12.309906+0000","last_peered":"2024-04-25T16:28:12.309906+0000","last_clean":"2024-04-25T16:28:12.309906+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:28:12.309906+0000","last_undegraded":"2024-04-25T16:28:12.309906+0000","last_fullsized":"2024-04-25T16:28:12.309906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.309787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.309787+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.309787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:42.695977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:36.698883+0000","last_change":"2024-04-25T16:26:56.564468+0000","last_active":"2024-04-25T16:27:36.698883+0000","last_peered":"2024-04-25T16:27:36.698883+0000","last_clean":"2024-04-25T16:27:36.698883+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:27:36.698883+0000","last_undegraded":"2024-04-25T16:27:36.698883+0000","last_fullsized":"2024-04-25T16:27:36.698883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:26:55.530379+0000","last_clean_scrub_stamp":"2024-04-25T16:26:55.530379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:07.480619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15613,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":31226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":884736,"data_stored":708870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":118010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":118046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":311296,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":286720,"data_stored":117920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":118080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":892928,"data_stored":708926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:28:30.331 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.332 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T16:26:43.492401+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=43, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.333 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.334 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.334 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.334 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.334 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.335 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.335 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.335 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.335 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.335 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.335 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.336 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.336 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.336 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.336 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.336 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.336 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T16:26:54.523761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-25T16:26:59.558966+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.337 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.338 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.339 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.339 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T16:26:55.530379+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=26, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T16:28:30.339 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T16:28:50.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:28:50.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.462+0000 7fe3473f3640 1 Processor -- start 2024-04-25T16:28:50.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.462+0000 7fe3473f3640 1 -- start start 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe3473f3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe34015d3e0 0x7fe3400b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe3473f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe34006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe3473f3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3400b1650 con 0x7fe3400b6970 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe3473f3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe340161620 con 0x7fe34015d3e0 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe3473f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe340160fd0 con 0x7fe340161ab0 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe34015d3e0 0x7fe3400b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe34015d3e0 0x7fe3400b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44276/0 (socket says 172.21.15.94:44276) 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 -- 172.21.15.94:0/4187842639 learned_addr learned my addr 172.21.15.94:0/4187842639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34498e640 1 --2- 172.21.15.94:0/4187842639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe34006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 -- 172.21.15.94:0/4187842639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 msgr2=0x7fe34006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 --2- 172.21.15.94:0/4187842639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe34006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 -- 172.21.15.94:0/4187842639 >> v1:172.21.15.138:6789/0 conn(0x7fe3400b6970 legacy=0x7fe3400b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.464+0000 7fe34518f640 1 -- 172.21.15.94:0/4187842639 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3401612b0 con 0x7fe34015d3e0 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.465+0000 7fe34518f640 1 --2- 172.21.15.94:0/4187842639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe3400b0d00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009e50 tx=0x7fe33c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f16ac9816ea4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.465+0000 7fe337fff640 1 -- 172.21.15.94:0/4187842639 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c036070 con 0x7fe34015d3e0 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.465+0000 7fe337fff640 1 -- 172.21.15.94:0/4187842639 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33c031600 con 0x7fe34015d3e0 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.465+0000 7fe337fff640 1 -- 172.21.15.94:0/4187842639 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0319d0 con 0x7fe34015d3e0 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 -- 172.21.15.94:0/4187842639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 msgr2=0x7fe3400b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 --2- 172.21.15.94:0/4187842639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe3400b0d00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009e50 tx=0x7fe33c02a6c0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 -- 172.21.15.94:0/4187842639 shutdown_connections 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 --2- 172.21.15.94:0/4187842639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe34006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 --2- 172.21.15.94:0/4187842639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe3400b0d00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 -- 172.21.15.94:0/4187842639 >> 172.21.15.94:0/4187842639 conn(0x7fe3400d23b0 msgr2=0x7fe340073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 -- 172.21.15.94:0/4187842639 shutdown_connections 2024-04-25T16:28:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.466+0000 7fe3473f3640 1 -- 172.21.15.94:0/4187842639 wait complete. 2024-04-25T16:28:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.468+0000 7fe3473f3640 1 Processor -- start 2024-04-25T16:28:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.468+0000 7fe3473f3640 1 -- start start 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe3473f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3400b6970 0x7fe3401f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe3473f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe34009cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe3473f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe3400d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe3473f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe34006dc40 con 0x7fe34015d3e0 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe3473f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe34006dac0 con 0x7fe3400b6970 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe3473f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe34006ddc0 con 0x7fe340161ab0 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe34498e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe34009cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe34498e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe34009cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44282/0 (socket says 172.21.15.94:44282) 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe34498e640 1 -- 172.21.15.94:0/2742073037 learned_addr learned my addr 172.21.15.94:0/2742073037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe34518f640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3400b6970 0x7fe3401f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.469+0000 7fe345990640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe3400d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe34498e640 1 -- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 msgr2=0x7fe3400d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe34498e640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe3400d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe34498e640 1 -- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3400b6970 msgr2=0x7fe3401f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe34498e640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3400b6970 0x7fe3401f7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe34498e640 1 -- 172.21.15.94:0/2742073037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400b1820 con 0x7fe34015d3e0 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe34498e640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe34009cff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe33000d620 tx=0x7fe33000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33000b620 con 0x7fe34015d3e0 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330019070 con 0x7fe34015d3e0 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3400b1ae0 con 0x7fe34015d3e0 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33000ba00 con 0x7fe34015d3e0 2024-04-25T16:28:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.470+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3400b20a0 con 0x7fe34015d3e0 2024-04-25T16:28:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.471+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34006a180 con 0x7fe34015d3e0 2024-04-25T16:28:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.472+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe33000bba0 con 0x7fe34015d3e0 2024-04-25T16:28:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.472+0000 7fe335ffb640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3240744b0 0x7fe3240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.472+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe330095880 con 0x7fe34015d3e0 2024-04-25T16:28:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.472+0000 7fe34518f640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3240744b0 0x7fe3240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.473+0000 7fe34518f640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3240744b0 0x7fe3240769c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe33c033000 tx=0x7fe33c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:50.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.477+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe33005f650 con 0x7fe34015d3e0 2024-04-25T16:28:50.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.647+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 --> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe34015cf90 con 0x7fe3240744b0 2024-04-25T16:28:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.655+0000 7fe335ffb640 1 -- 172.21.15.94:0/2742073037 <== mgr.4101 v2:172.21.15.94:6832/2607833826 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332588 (secure 0 0 0) 0x7fe34015cf90 con 0x7fe3240744b0 2024-04-25T16:28:50.660 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:28:50.662 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:28:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3240744b0 msgr2=0x7fe3240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3240744b0 0x7fe3240769c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe33c033000 tx=0x7fe33c009000 comp rx=0 tx=0).stop 2024-04-25T16:28:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 msgr2=0x7fe34009cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe34009cff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe33000d620 tx=0x7fe33000dae0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 shutdown_connections 2024-04-25T16:28:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fe3240744b0 0x7fe3240769c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe340161ab0 0x7fe3400d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe34015d3e0 0x7fe34009cff0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 --2- 172.21.15.94:0/2742073037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3400b6970 0x7fe3401f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 >> 172.21.15.94:0/2742073037 conn(0x7fe3400d23b0 msgr2=0x7fe34006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:50.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 shutdown_connections 2024-04-25T16:28:50.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.659+0000 7fe3473f3640 1 -- 172.21.15.94:0/2742073037 wait complete. 2024-04-25T16:28:50.670 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-25T16:28:49.326945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605981,"num_objects":25,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":169,"num_write_kb":1668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236256,"kb_used_data":3488,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738464,"statfs":{"total":773094113280,"available":772852187136,"internally_reserved":0,"allocated":3571712,"data_stored":2155322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004310"},"pg_stats":[{"pgid":"2.1f","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:01.512673+0000","last_change":"2024-04-25T16:28:01.512707+0000","last_active":"2024-04-25T16:28:01.512673+0000","last_peered":"2024-04-25T16:28:01.512673+0000","last_clean":"2024-04-25T16:28:01.512673+0000","last_became_active":"2024-04-25T16:26:55.557627+0000","last_became_peered":"2024-04-25T16:26:55.557627+0000","last_unstale":"2024-04-25T16:28:01.512673+0000","last_undegraded":"2024-04-25T16:28:01.512673+0000","last_fullsized":"2024-04-25T16:28:01.512673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-25T16:28:01.512572+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-25T16:28:01.512572+0000","last_clean_scrub_stamp":"2024-04-25T16:28:01.512572+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:31.188129+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.671143+0000","last_change":"2024-04-25T16:28:12.671186+0000","last_active":"2024-04-25T16:28:12.671143+0000","last_peered":"2024-04-25T16:28:12.671143+0000","last_clean":"2024-04-25T16:28:12.671143+0000","last_became_active":"2024-04-25T16:26:56.561459+0000","last_became_peered":"2024-04-25T16:26:56.561459+0000","last_unstale":"2024-04-25T16:28:12.671143+0000","last_undegraded":"2024-04-25T16:28:12.671143+0000","last_fullsized":"2024-04-25T16:28:12.671143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.671030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.671030+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.671030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:14.544960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:31.687520+0000","last_change":"2024-04-25T16:28:31.687581+0000","last_active":"2024-04-25T16:28:31.687520+0000","last_peered":"2024-04-25T16:28:31.687520+0000","last_clean":"2024-04-25T16:28:31.687520+0000","last_became_active":"2024-04-25T16:27:00.607087+0000","last_became_peered":"2024-04-25T16:27:00.607087+0000","last_unstale":"2024-04-25T16:28:31.687520+0000","last_undegraded":"2024-04-25T16:28:31.687520+0000","last_fullsized":"2024-04-25T16:28:31.687520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:31.687393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:31.687393+0000","last_clean_scrub_stamp":"2024-04-25T16:28:31.687393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.051441+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:18.718029+0000","last_change":"2024-04-25T16:28:18.718083+0000","last_active":"2024-04-25T16:28:18.718029+0000","last_peered":"2024-04-25T16:28:18.718029+0000","last_clean":"2024-04-25T16:28:18.718029+0000","last_became_active":"2024-04-25T16:26:55.556583+0000","last_became_peered":"2024-04-25T16:26:55.556583+0000","last_unstale":"2024-04-25T16:28:18.718029+0000","last_undegraded":"2024-04-25T16:28:18.718029+0000","last_fullsized":"2024-04-25T16:28:18.718029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T16:28:18.717906+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T16:28:18.717906+0000","last_clean_scrub_stamp":"2024-04-25T16:28:18.717906+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:51.102949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:27:59.464325+0000","last_change":"2024-04-25T16:27:59.464351+0000","last_active":"2024-04-25T16:27:59.464325+0000","last_peered":"2024-04-25T16:27:59.464325+0000","last_clean":"2024-04-25T16:27:59.464325+0000","last_became_active":"2024-04-25T16:26:56.557608+0000","last_became_peered":"2024-04-25T16:26:56.557608+0000","last_unstale":"2024-04-25T16:27:59.464325+0000","last_undegraded":"2024-04-25T16:27:59.464325+0000","last_fullsized":"2024-04-25T16:27:59.464325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:27:59.464232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:27:59.464232+0000","last_clean_scrub_stamp":"2024-04-25T16:27:59.464232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:51.004937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:25.769219+0000","last_change":"2024-04-25T16:28:25.769251+0000","last_active":"2024-04-25T16:28:25.769219+0000","last_peered":"2024-04-25T16:28:25.769219+0000","last_clean":"2024-04-25T16:28:25.769219+0000","last_became_active":"2024-04-25T16:27:00.588908+0000","last_became_peered":"2024-04-25T16:27:00.588908+0000","last_unstale":"2024-04-25T16:28:25.769219+0000","last_undegraded":"2024-04-25T16:28:25.769219+0000","last_fullsized":"2024-04-25T16:28:25.769219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:25.769123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:25.769123+0000","last_clean_scrub_stamp":"2024-04-25T16:28:25.769123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:56.155313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:35.118121+0000","last_change":"2024-04-25T16:28:35.118156+0000","last_active":"2024-04-25T16:28:35.118121+0000","last_peered":"2024-04-25T16:28:35.118121+0000","last_clean":"2024-04-25T16:28:35.118121+0000","last_became_active":"2024-04-25T16:26:55.559408+0000","last_became_peered":"2024-04-25T16:26:55.559408+0000","last_unstale":"2024-04-25T16:28:35.118121+0000","last_undegraded":"2024-04-25T16:28:35.118121+0000","last_fullsized":"2024-04-25T16:28:35.118121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:35.118020+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:35.118020+0000","last_clean_scrub_stamp":"2024-04-25T16:28:35.118020+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:47.238965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:28.950984+0000","last_change":"2024-04-25T16:28:28.951029+0000","last_active":"2024-04-25T16:28:28.950984+0000","last_peered":"2024-04-25T16:28:28.950984+0000","last_clean":"2024-04-25T16:28:28.950984+0000","last_became_active":"2024-04-25T16:26:56.562090+0000","last_became_peered":"2024-04-25T16:26:56.562090+0000","last_unstale":"2024-04-25T16:28:28.950984+0000","last_undegraded":"2024-04-25T16:28:28.950984+0000","last_fullsized":"2024-04-25T16:28:28.950984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:28.950862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:28.950862+0000","last_clean_scrub_stamp":"2024-04-25T16:28:28.950862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:18.474409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:23.849952+0000","last_change":"2024-04-25T16:28:23.849984+0000","last_active":"2024-04-25T16:28:23.849952+0000","last_peered":"2024-04-25T16:28:23.849952+0000","last_clean":"2024-04-25T16:28:23.849952+0000","last_became_active":"2024-04-25T16:27:00.585713+0000","last_became_peered":"2024-04-25T16:27:00.585713+0000","last_unstale":"2024-04-25T16:28:23.849952+0000","last_undegraded":"2024-04-25T16:28:23.849952+0000","last_fullsized":"2024-04-25T16:28:23.849952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:23.849835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:23.849835+0000","last_clean_scrub_stamp":"2024-04-25T16:28:23.849835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:56.903111+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:38.138947+0000","last_change":"2024-04-25T16:28:38.138981+0000","last_active":"2024-04-25T16:28:38.138947+0000","last_peered":"2024-04-25T16:28:38.138947+0000","last_clean":"2024-04-25T16:28:38.138947+0000","last_became_active":"2024-04-25T16:26:55.561659+0000","last_became_peered":"2024-04-25T16:26:55.561659+0000","last_unstale":"2024-04-25T16:28:38.138947+0000","last_undegraded":"2024-04-25T16:28:38.138947+0000","last_fullsized":"2024-04-25T16:28:38.138947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:38.138840+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:38.138840+0000","last_clean_scrub_stamp":"2024-04-25T16:28:38.138840+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:58:06.637330+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:23.084686+0000","last_change":"2024-04-25T16:28:23.084731+0000","last_active":"2024-04-25T16:28:23.084686+0000","last_peered":"2024-04-25T16:28:23.084686+0000","last_clean":"2024-04-25T16:28:23.084686+0000","last_became_active":"2024-04-25T16:26:56.564485+0000","last_became_peered":"2024-04-25T16:26:56.564485+0000","last_unstale":"2024-04-25T16:28:23.084686+0000","last_undegraded":"2024-04-25T16:28:23.084686+0000","last_fullsized":"2024-04-25T16:28:23.084686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:23.084560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:23.084560+0000","last_clean_scrub_stamp":"2024-04-25T16:28:23.084560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:24.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:24.810513+0000","last_change":"2024-04-25T16:28:24.810583+0000","last_active":"2024-04-25T16:28:24.810513+0000","last_peered":"2024-04-25T16:28:24.810513+0000","last_clean":"2024-04-25T16:28:24.810513+0000","last_became_active":"2024-04-25T16:27:00.587989+0000","last_became_peered":"2024-04-25T16:27:00.587989+0000","last_unstale":"2024-04-25T16:28:24.810513+0000","last_undegraded":"2024-04-25T16:28:24.810513+0000","last_fullsized":"2024-04-25T16:28:24.810513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:24.810405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:24.810405+0000","last_clean_scrub_stamp":"2024-04-25T16:28:24.810405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:54.863780+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:31.195009+0000","last_change":"2024-04-25T16:28:31.195034+0000","last_active":"2024-04-25T16:28:31.195009+0000","last_peered":"2024-04-25T16:28:31.195009+0000","last_clean":"2024-04-25T16:28:31.195009+0000","last_became_active":"2024-04-25T16:26:56.550932+0000","last_became_peered":"2024-04-25T16:26:56.550932+0000","last_unstale":"2024-04-25T16:28:31.195009+0000","last_undegraded":"2024-04-25T16:28:31.195009+0000","last_fullsized":"2024-04-25T16:28:31.195009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:31.194910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:31.194910+0000","last_clean_scrub_stamp":"2024-04-25T16:28:31.194910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:35.600833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:34.133746+0000","last_change":"2024-04-25T16:28:34.133779+0000","last_active":"2024-04-25T16:28:34.133746+0000","last_peered":"2024-04-25T16:28:34.133746+0000","last_clean":"2024-04-25T16:28:34.133746+0000","last_became_active":"2024-04-25T16:26:55.559672+0000","last_became_peered":"2024-04-25T16:26:55.559672+0000","last_unstale":"2024-04-25T16:28:34.133746+0000","last_undegraded":"2024-04-25T16:28:34.133746+0000","last_fullsized":"2024-04-25T16:28:34.133746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:34.133655+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:34.133655+0000","last_clean_scrub_stamp":"2024-04-25T16:28:34.133655+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:06.518698+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:29.865235+0000","last_change":"2024-04-25T16:28:29.865278+0000","last_active":"2024-04-25T16:28:29.865235+0000","last_peered":"2024-04-25T16:28:29.865235+0000","last_clean":"2024-04-25T16:28:29.865235+0000","last_became_active":"2024-04-25T16:27:00.590845+0000","last_became_peered":"2024-04-25T16:27:00.590845+0000","last_unstale":"2024-04-25T16:28:29.865235+0000","last_undegraded":"2024-04-25T16:28:29.865235+0000","last_fullsized":"2024-04-25T16:28:29.865235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:29.865123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:29.865123+0000","last_clean_scrub_stamp":"2024-04-25T16:28:29.865123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:19.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:23.666726+0000","last_change":"2024-04-25T16:28:23.666752+0000","last_active":"2024-04-25T16:28:23.666726+0000","last_peered":"2024-04-25T16:28:23.666726+0000","last_clean":"2024-04-25T16:28:23.666726+0000","last_became_active":"2024-04-25T16:26:56.553149+0000","last_became_peered":"2024-04-25T16:26:56.553149+0000","last_unstale":"2024-04-25T16:28:23.666726+0000","last_undegraded":"2024-04-25T16:28:23.666726+0000","last_fullsized":"2024-04-25T16:28:23.666726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:23.666628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:23.666628+0000","last_clean_scrub_stamp":"2024-04-25T16:28:23.666628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:01.002536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:04.469988+0000","last_change":"2024-04-25T16:28:04.470030+0000","last_active":"2024-04-25T16:28:04.469988+0000","last_peered":"2024-04-25T16:28:04.469988+0000","last_clean":"2024-04-25T16:28:04.469988+0000","last_became_active":"2024-04-25T16:26:55.551501+0000","last_became_peered":"2024-04-25T16:26:55.551501+0000","last_unstale":"2024-04-25T16:28:04.469988+0000","last_undegraded":"2024-04-25T16:28:04.469988+0000","last_fullsized":"2024-04-25T16:28:04.469988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:04.469891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:04.469891+0000","last_clean_scrub_stamp":"2024-04-25T16:28:04.469891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:12.364601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.790684+0000","last_change":"2024-04-25T16:28:26.790712+0000","last_active":"2024-04-25T16:28:26.790684+0000","last_peered":"2024-04-25T16:28:26.790684+0000","last_clean":"2024-04-25T16:28:26.790684+0000","last_became_active":"2024-04-25T16:27:00.589042+0000","last_became_peered":"2024-04-25T16:27:00.589042+0000","last_unstale":"2024-04-25T16:28:26.790684+0000","last_undegraded":"2024-04-25T16:28:26.790684+0000","last_fullsized":"2024-04-25T16:28:26.790684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.790575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.790575+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.790575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.054670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:20.715683+0000","last_change":"2024-04-25T16:28:20.715727+0000","last_active":"2024-04-25T16:28:20.715683+0000","last_peered":"2024-04-25T16:28:20.715683+0000","last_clean":"2024-04-25T16:28:20.715683+0000","last_became_active":"2024-04-25T16:26:56.550280+0000","last_became_peered":"2024-04-25T16:26:56.550280+0000","last_unstale":"2024-04-25T16:28:20.715683+0000","last_undegraded":"2024-04-25T16:28:20.715683+0000","last_fullsized":"2024-04-25T16:28:20.715683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:20.715547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:20.715547+0000","last_clean_scrub_stamp":"2024-04-25T16:28:20.715547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:54.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:09.384028+0000","last_change":"2024-04-25T16:28:09.384108+0000","last_active":"2024-04-25T16:28:09.384028+0000","last_peered":"2024-04-25T16:28:09.384028+0000","last_clean":"2024-04-25T16:28:09.384028+0000","last_became_active":"2024-04-25T16:26:55.560784+0000","last_became_peered":"2024-04-25T16:26:55.560784+0000","last_unstale":"2024-04-25T16:28:09.384028+0000","last_undegraded":"2024-04-25T16:28:09.384028+0000","last_fullsized":"2024-04-25T16:28:09.384028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:09.383913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:09.383913+0000","last_clean_scrub_stamp":"2024-04-25T16:28:09.383913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:10.807069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.386653+0000","last_change":"2024-04-25T16:28:17.386677+0000","last_active":"2024-04-25T16:28:17.386653+0000","last_peered":"2024-04-25T16:28:17.386653+0000","last_clean":"2024-04-25T16:28:17.386653+0000","last_became_active":"2024-04-25T16:27:36.751463+0000","last_became_peered":"2024-04-25T16:27:36.751463+0000","last_unstale":"2024-04-25T16:28:17.386653+0000","last_undegraded":"2024-04-25T16:28:17.386653+0000","last_fullsized":"2024-04-25T16:28:17.386653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:17.386580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:17.386580+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.386580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:19.467904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:18.829186+0000","last_change":"2024-04-25T16:28:18.829238+0000","last_active":"2024-04-25T16:28:18.829186+0000","last_peered":"2024-04-25T16:28:18.829186+0000","last_clean":"2024-04-25T16:28:18.829186+0000","last_became_active":"2024-04-25T16:26:56.550360+0000","last_became_peered":"2024-04-25T16:26:56.550360+0000","last_unstale":"2024-04-25T16:28:18.829186+0000","last_undegraded":"2024-04-25T16:28:18.829186+0000","last_fullsized":"2024-04-25T16:28:18.829186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:18.829066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:18.829066+0000","last_clean_scrub_stamp":"2024-04-25T16:28:18.829066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:48.504622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:36.164763+0000","last_change":"2024-04-25T16:28:36.164791+0000","last_active":"2024-04-25T16:28:36.164763+0000","last_peered":"2024-04-25T16:28:36.164763+0000","last_clean":"2024-04-25T16:28:36.164763+0000","last_became_active":"2024-04-25T16:26:55.560666+0000","last_became_peered":"2024-04-25T16:26:55.560666+0000","last_unstale":"2024-04-25T16:28:36.164763+0000","last_undegraded":"2024-04-25T16:28:36.164763+0000","last_fullsized":"2024-04-25T16:28:36.164763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:36.164639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:36.164639+0000","last_clean_scrub_stamp":"2024-04-25T16:28:36.164639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:25.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:22.895366+0000","last_change":"2024-04-25T16:28:22.895399+0000","last_active":"2024-04-25T16:28:22.895366+0000","last_peered":"2024-04-25T16:28:22.895366+0000","last_clean":"2024-04-25T16:28:22.895366+0000","last_became_active":"2024-04-25T16:27:00.585828+0000","last_became_peered":"2024-04-25T16:27:00.585828+0000","last_unstale":"2024-04-25T16:28:22.895366+0000","last_undegraded":"2024-04-25T16:28:22.895366+0000","last_fullsized":"2024-04-25T16:28:22.895366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:22.895238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:22.895238+0000","last_clean_scrub_stamp":"2024-04-25T16:28:22.895238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:24.318394+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:04.584950+0000","last_change":"2024-04-25T16:28:04.584991+0000","last_active":"2024-04-25T16:28:04.584950+0000","last_peered":"2024-04-25T16:28:04.584950+0000","last_clean":"2024-04-25T16:28:04.584950+0000","last_became_active":"2024-04-25T16:26:56.551225+0000","last_became_peered":"2024-04-25T16:26:56.551225+0000","last_unstale":"2024-04-25T16:28:04.584950+0000","last_undegraded":"2024-04-25T16:28:04.584950+0000","last_fullsized":"2024-04-25T16:28:04.584950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:04.584863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:04.584863+0000","last_clean_scrub_stamp":"2024-04-25T16:28:04.584863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:52.800183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:29.134057+0000","last_change":"2024-04-25T16:28:29.134112+0000","last_active":"2024-04-25T16:28:29.134057+0000","last_peered":"2024-04-25T16:28:29.134057+0000","last_clean":"2024-04-25T16:28:29.134057+0000","last_became_active":"2024-04-25T16:26:55.556729+0000","last_became_peered":"2024-04-25T16:26:55.556729+0000","last_unstale":"2024-04-25T16:28:29.134057+0000","last_undegraded":"2024-04-25T16:28:29.134057+0000","last_fullsized":"2024-04-25T16:28:29.134057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:29.133927+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:29.133927+0000","last_clean_scrub_stamp":"2024-04-25T16:28:29.133927+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:37.212894+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:30.827504+0000","last_change":"2024-04-25T16:28:30.827564+0000","last_active":"2024-04-25T16:28:30.827504+0000","last_peered":"2024-04-25T16:28:30.827504+0000","last_clean":"2024-04-25T16:28:30.827504+0000","last_became_active":"2024-04-25T16:27:00.602901+0000","last_became_peered":"2024-04-25T16:27:00.602901+0000","last_unstale":"2024-04-25T16:28:30.827504+0000","last_undegraded":"2024-04-25T16:28:30.827504+0000","last_fullsized":"2024-04-25T16:28:30.827504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:30.827391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:30.827391+0000","last_clean_scrub_stamp":"2024-04-25T16:28:30.827391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:55.834935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:25.674010+0000","last_change":"2024-04-25T16:28:25.674052+0000","last_active":"2024-04-25T16:28:25.674010+0000","last_peered":"2024-04-25T16:28:25.674010+0000","last_clean":"2024-04-25T16:28:25.674010+0000","last_became_active":"2024-04-25T16:26:56.568842+0000","last_became_peered":"2024-04-25T16:26:56.568842+0000","last_unstale":"2024-04-25T16:28:25.674010+0000","last_undegraded":"2024-04-25T16:28:25.674010+0000","last_fullsized":"2024-04-25T16:28:25.674010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:25.673891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:25.673891+0000","last_clean_scrub_stamp":"2024-04-25T16:28:25.673891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:09.763480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:07.420778+0000","last_change":"2024-04-25T16:28:07.420852+0000","last_active":"2024-04-25T16:28:07.420778+0000","last_peered":"2024-04-25T16:28:07.420778+0000","last_clean":"2024-04-25T16:28:07.420778+0000","last_became_active":"2024-04-25T16:26:55.553831+0000","last_became_peered":"2024-04-25T16:26:55.553831+0000","last_unstale":"2024-04-25T16:28:07.420778+0000","last_undegraded":"2024-04-25T16:28:07.420778+0000","last_fullsized":"2024-04-25T16:28:07.420778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:07.420639+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:07.420639+0000","last_clean_scrub_stamp":"2024-04-25T16:28:07.420639+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:19.658768+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:28.703201+0000","last_change":"2024-04-25T16:28:28.703252+0000","last_active":"2024-04-25T16:28:28.703201+0000","last_peered":"2024-04-25T16:28:28.703201+0000","last_clean":"2024-04-25T16:28:28.703201+0000","last_became_active":"2024-04-25T16:27:00.601921+0000","last_became_peered":"2024-04-25T16:27:00.601921+0000","last_unstale":"2024-04-25T16:28:28.703201+0000","last_undegraded":"2024-04-25T16:28:28.703201+0000","last_fullsized":"2024-04-25T16:28:28.703201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:28.703054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:28.703054+0000","last_clean_scrub_stamp":"2024-04-25T16:28:28.703054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:56.281277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:11.308379+0000","last_change":"2024-04-25T16:28:11.308407+0000","last_active":"2024-04-25T16:28:11.308379+0000","last_peered":"2024-04-25T16:28:11.308379+0000","last_clean":"2024-04-25T16:28:11.308379+0000","last_became_active":"2024-04-25T16:26:56.545251+0000","last_became_peered":"2024-04-25T16:26:56.545251+0000","last_unstale":"2024-04-25T16:28:11.308379+0000","last_undegraded":"2024-04-25T16:28:11.308379+0000","last_fullsized":"2024-04-25T16:28:11.308379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:11.308277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:11.308277+0000","last_clean_scrub_stamp":"2024-04-25T16:28:11.308277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:41.361653+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:33.141248+0000","last_change":"2024-04-25T16:28:33.141301+0000","last_active":"2024-04-25T16:28:33.141248+0000","last_peered":"2024-04-25T16:28:33.141248+0000","last_clean":"2024-04-25T16:28:33.141248+0000","last_became_active":"2024-04-25T16:26:55.558972+0000","last_became_peered":"2024-04-25T16:26:55.558972+0000","last_unstale":"2024-04-25T16:28:33.141248+0000","last_undegraded":"2024-04-25T16:28:33.141248+0000","last_fullsized":"2024-04-25T16:28:33.141248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:33.141120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:33.141120+0000","last_clean_scrub_stamp":"2024-04-25T16:28:33.141120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:10.676881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.540825+0000","last_change":"2024-04-25T16:28:13.540871+0000","last_active":"2024-04-25T16:28:13.540825+0000","last_peered":"2024-04-25T16:28:13.540825+0000","last_clean":"2024-04-25T16:28:13.540825+0000","last_became_active":"2024-04-25T16:27:00.601914+0000","last_became_peered":"2024-04-25T16:27:00.601914+0000","last_unstale":"2024-04-25T16:28:13.540825+0000","last_undegraded":"2024-04-25T16:28:13.540825+0000","last_fullsized":"2024-04-25T16:28:13.540825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.540729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.540729+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.540729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:42.039219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:30.999188+0000","last_change":"2024-04-25T16:28:30.999240+0000","last_active":"2024-04-25T16:28:30.999188+0000","last_peered":"2024-04-25T16:28:30.999188+0000","last_clean":"2024-04-25T16:28:30.999188+0000","last_became_active":"2024-04-25T16:26:56.563702+0000","last_became_peered":"2024-04-25T16:26:56.563702+0000","last_unstale":"2024-04-25T16:28:30.999188+0000","last_undegraded":"2024-04-25T16:28:30.999188+0000","last_fullsized":"2024-04-25T16:28:30.999188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:30.999081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:30.999081+0000","last_clean_scrub_stamp":"2024-04-25T16:28:30.999081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:59.497572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:06.422639+0000","last_change":"2024-04-25T16:28:06.422702+0000","last_active":"2024-04-25T16:28:06.422639+0000","last_peered":"2024-04-25T16:28:06.422639+0000","last_clean":"2024-04-25T16:28:06.422639+0000","last_became_active":"2024-04-25T16:26:55.553152+0000","last_became_peered":"2024-04-25T16:26:55.553152+0000","last_unstale":"2024-04-25T16:28:06.422639+0000","last_undegraded":"2024-04-25T16:28:06.422639+0000","last_fullsized":"2024-04-25T16:28:06.422639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:06.422514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:06.422514+0000","last_clean_scrub_stamp":"2024-04-25T16:28:06.422514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:08.503838+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:18.386021+0000","last_change":"2024-04-25T16:28:18.386048+0000","last_active":"2024-04-25T16:28:18.386021+0000","last_peered":"2024-04-25T16:28:18.386021+0000","last_clean":"2024-04-25T16:28:18.386021+0000","last_became_active":"2024-04-25T16:27:00.588497+0000","last_became_peered":"2024-04-25T16:27:00.588497+0000","last_unstale":"2024-04-25T16:28:18.386021+0000","last_undegraded":"2024-04-25T16:28:18.386021+0000","last_fullsized":"2024-04-25T16:28:18.386021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:18.385937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:18.385937+0000","last_clean_scrub_stamp":"2024-04-25T16:28:18.385937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:46.884427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:20.901109+0000","last_change":"2024-04-25T16:28:20.901136+0000","last_active":"2024-04-25T16:28:20.901109+0000","last_peered":"2024-04-25T16:28:20.901109+0000","last_clean":"2024-04-25T16:28:20.901109+0000","last_became_active":"2024-04-25T16:26:56.553192+0000","last_became_peered":"2024-04-25T16:26:56.553192+0000","last_unstale":"2024-04-25T16:28:20.901109+0000","last_undegraded":"2024-04-25T16:28:20.901109+0000","last_fullsized":"2024-04-25T16:28:20.901109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:20.901004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:20.901004+0000","last_clean_scrub_stamp":"2024-04-25T16:28:20.901004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:33.139133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:25.066954+0000","last_change":"2024-04-25T16:28:25.066982+0000","last_active":"2024-04-25T16:28:25.066954+0000","last_peered":"2024-04-25T16:28:25.066954+0000","last_clean":"2024-04-25T16:28:25.066954+0000","last_became_active":"2024-04-25T16:26:55.553721+0000","last_became_peered":"2024-04-25T16:26:55.553721+0000","last_unstale":"2024-04-25T16:28:25.066954+0000","last_undegraded":"2024-04-25T16:28:25.066954+0000","last_fullsized":"2024-04-25T16:28:25.066954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:25.066856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:25.066856+0000","last_clean_scrub_stamp":"2024-04-25T16:28:25.066856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:55.453046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:16.382966+0000","last_change":"2024-04-25T16:28:16.382994+0000","last_active":"2024-04-25T16:28:16.382966+0000","last_peered":"2024-04-25T16:28:16.382966+0000","last_clean":"2024-04-25T16:28:16.382966+0000","last_became_active":"2024-04-25T16:27:00.588403+0000","last_became_peered":"2024-04-25T16:27:00.588403+0000","last_unstale":"2024-04-25T16:28:16.382966+0000","last_undegraded":"2024-04-25T16:28:16.382966+0000","last_fullsized":"2024-04-25T16:28:16.382966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:16.382863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:16.382863+0000","last_clean_scrub_stamp":"2024-04-25T16:28:16.382863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:05.906410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:10.338608+0000","last_change":"2024-04-25T16:28:10.338634+0000","last_active":"2024-04-25T16:28:10.338608+0000","last_peered":"2024-04-25T16:28:10.338608+0000","last_clean":"2024-04-25T16:28:10.338608+0000","last_became_active":"2024-04-25T16:26:56.543081+0000","last_became_peered":"2024-04-25T16:26:56.543081+0000","last_unstale":"2024-04-25T16:28:10.338608+0000","last_undegraded":"2024-04-25T16:28:10.338608+0000","last_fullsized":"2024-04-25T16:28:10.338608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:10.338507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:10.338507+0000","last_clean_scrub_stamp":"2024-04-25T16:28:10.338507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:43.391783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:10.476816+0000","last_change":"2024-04-25T16:28:10.476861+0000","last_active":"2024-04-25T16:28:10.476816+0000","last_peered":"2024-04-25T16:28:10.476816+0000","last_clean":"2024-04-25T16:28:10.476816+0000","last_became_active":"2024-04-25T16:26:55.551125+0000","last_became_peered":"2024-04-25T16:26:55.551125+0000","last_unstale":"2024-04-25T16:28:10.476816+0000","last_undegraded":"2024-04-25T16:28:10.476816+0000","last_fullsized":"2024-04-25T16:28:10.476816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:10.476675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:10.476675+0000","last_clean_scrub_stamp":"2024-04-25T16:28:10.476675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:18:52.884389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.339976+0000","last_change":"2024-04-25T16:28:17.340002+0000","last_active":"2024-04-25T16:28:17.339976+0000","last_peered":"2024-04-25T16:28:17.339976+0000","last_clean":"2024-04-25T16:28:17.339976+0000","last_became_active":"2024-04-25T16:27:00.589452+0000","last_became_peered":"2024-04-25T16:27:00.589452+0000","last_unstale":"2024-04-25T16:28:17.339976+0000","last_undegraded":"2024-04-25T16:28:17.339976+0000","last_fullsized":"2024-04-25T16:28:17.339976+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:17.339888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:17.339888+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.339888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:42.347432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:06.566517+0000","last_change":"2024-04-25T16:28:06.566566+0000","last_active":"2024-04-25T16:28:06.566517+0000","last_peered":"2024-04-25T16:28:06.566517+0000","last_clean":"2024-04-25T16:28:06.566517+0000","last_became_active":"2024-04-25T16:26:56.555459+0000","last_became_peered":"2024-04-25T16:26:56.555459+0000","last_unstale":"2024-04-25T16:28:06.566517+0000","last_undegraded":"2024-04-25T16:28:06.566517+0000","last_fullsized":"2024-04-25T16:28:06.566517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:06.566415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:06.566415+0000","last_clean_scrub_stamp":"2024-04-25T16:28:06.566415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:10.972339+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:37.148036+0000","last_change":"2024-04-25T16:28:37.148064+0000","last_active":"2024-04-25T16:28:37.148036+0000","last_peered":"2024-04-25T16:28:37.148036+0000","last_clean":"2024-04-25T16:28:37.148036+0000","last_became_active":"2024-04-25T16:26:55.560204+0000","last_became_peered":"2024-04-25T16:26:55.560204+0000","last_unstale":"2024-04-25T16:28:37.148036+0000","last_undegraded":"2024-04-25T16:28:37.148036+0000","last_fullsized":"2024-04-25T16:28:37.148036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:37.147934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:37.147934+0000","last_clean_scrub_stamp":"2024-04-25T16:28:37.147934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:02.155478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.545485+0000","last_change":"2024-04-25T16:28:14.545529+0000","last_active":"2024-04-25T16:28:14.545485+0000","last_peered":"2024-04-25T16:28:14.545485+0000","last_clean":"2024-04-25T16:28:14.545485+0000","last_became_active":"2024-04-25T16:27:00.601908+0000","last_became_peered":"2024-04-25T16:27:00.601908+0000","last_unstale":"2024-04-25T16:28:14.545485+0000","last_undegraded":"2024-04-25T16:28:14.545485+0000","last_fullsized":"2024-04-25T16:28:14.545485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.545387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.545387+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.545387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:37.777713+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.738461+0000","last_change":"2024-04-25T16:28:15.738503+0000","last_active":"2024-04-25T16:28:15.738461+0000","last_peered":"2024-04-25T16:28:15.738461+0000","last_clean":"2024-04-25T16:28:15.738461+0000","last_became_active":"2024-04-25T16:26:56.564390+0000","last_became_peered":"2024-04-25T16:26:56.564390+0000","last_unstale":"2024-04-25T16:28:15.738461+0000","last_undegraded":"2024-04-25T16:28:15.738461+0000","last_fullsized":"2024-04-25T16:28:15.738461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.738268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.738268+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.738268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:45.791644+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:24.089835+0000","last_change":"2024-04-25T16:28:24.089894+0000","last_active":"2024-04-25T16:28:24.089835+0000","last_peered":"2024-04-25T16:28:24.089835+0000","last_clean":"2024-04-25T16:28:24.089835+0000","last_became_active":"2024-04-25T16:26:55.553517+0000","last_became_peered":"2024-04-25T16:26:55.553517+0000","last_unstale":"2024-04-25T16:28:24.089835+0000","last_undegraded":"2024-04-25T16:28:24.089835+0000","last_fullsized":"2024-04-25T16:28:24.089835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:24.089728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:24.089728+0000","last_clean_scrub_stamp":"2024-04-25T16:28:24.089728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:12.068224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'74","reported_seq":134,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:32.166095+0000","last_change":"2024-04-25T16:28:32.166095+0000","last_active":"2024-04-25T16:28:32.166095+0000","last_peered":"2024-04-25T16:28:32.166095+0000","last_clean":"2024-04-25T16:28:32.166095+0000","last_became_active":"2024-04-25T16:26:44.598573+0000","last_became_peered":"2024-04-25T16:26:44.598573+0000","last_unstale":"2024-04-25T16:28:32.166095+0000","last_undegraded":"2024-04-25T16:28:32.166095+0000","last_fullsized":"2024-04-25T16:28:32.166095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'74","last_scrub_stamp":"2024-04-25T16:28:32.165948+0000","last_deep_scrub":"27'74","last_deep_scrub_stamp":"2024-04-25T16:28:32.165948+0000","last_clean_scrub_stamp":"2024-04-25T16:28:32.165948+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:51.384627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:39.186387+0000","last_change":"2024-04-25T16:28:39.186415+0000","last_active":"2024-04-25T16:28:39.186387+0000","last_peered":"2024-04-25T16:28:39.186387+0000","last_clean":"2024-04-25T16:28:39.186387+0000","last_became_active":"2024-04-25T16:27:00.604710+0000","last_became_peered":"2024-04-25T16:27:00.604710+0000","last_unstale":"2024-04-25T16:28:39.186387+0000","last_undegraded":"2024-04-25T16:28:39.186387+0000","last_fullsized":"2024-04-25T16:28:39.186387+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:39.186286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:39.186286+0000","last_clean_scrub_stamp":"2024-04-25T16:28:39.186286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.418612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:46.268050+0000","last_change":"2024-04-25T16:28:46.268078+0000","last_active":"2024-04-25T16:28:46.268050+0000","last_peered":"2024-04-25T16:28:46.268050+0000","last_clean":"2024-04-25T16:28:46.268050+0000","last_became_active":"2024-04-25T16:26:56.571721+0000","last_became_peered":"2024-04-25T16:26:56.571721+0000","last_unstale":"2024-04-25T16:28:46.268050+0000","last_undegraded":"2024-04-25T16:28:46.268050+0000","last_fullsized":"2024-04-25T16:28:46.268050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:46.267951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:46.267951+0000","last_clean_scrub_stamp":"2024-04-25T16:28:46.267951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:31.296134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:11.500122+0000","last_change":"2024-04-25T16:28:11.500162+0000","last_active":"2024-04-25T16:28:11.500122+0000","last_peered":"2024-04-25T16:28:11.500122+0000","last_clean":"2024-04-25T16:28:11.500122+0000","last_became_active":"2024-04-25T16:26:55.561571+0000","last_became_peered":"2024-04-25T16:26:55.561571+0000","last_unstale":"2024-04-25T16:28:11.500122+0000","last_undegraded":"2024-04-25T16:28:11.500122+0000","last_fullsized":"2024-04-25T16:28:11.500122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:11.500002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:11.500002+0000","last_clean_scrub_stamp":"2024-04-25T16:28:11.500002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:42.358174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:38.180140+0000","last_change":"2024-04-25T16:28:38.180169+0000","last_active":"2024-04-25T16:28:38.180140+0000","last_peered":"2024-04-25T16:28:38.180140+0000","last_clean":"2024-04-25T16:28:38.180140+0000","last_became_active":"2024-04-25T16:27:00.601857+0000","last_became_peered":"2024-04-25T16:27:00.601857+0000","last_unstale":"2024-04-25T16:28:38.180140+0000","last_undegraded":"2024-04-25T16:28:38.180140+0000","last_fullsized":"2024-04-25T16:28:38.180140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:38.180032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:38.180032+0000","last_clean_scrub_stamp":"2024-04-25T16:28:38.180032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:58:06.678523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:32.021345+0000","last_change":"2024-04-25T16:28:32.021374+0000","last_active":"2024-04-25T16:28:32.021345+0000","last_peered":"2024-04-25T16:28:32.021345+0000","last_clean":"2024-04-25T16:28:32.021345+0000","last_became_active":"2024-04-25T16:26:56.565011+0000","last_became_peered":"2024-04-25T16:26:56.565011+0000","last_unstale":"2024-04-25T16:28:32.021345+0000","last_undegraded":"2024-04-25T16:28:32.021345+0000","last_fullsized":"2024-04-25T16:28:32.021345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:32.021230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:32.021230+0000","last_clean_scrub_stamp":"2024-04-25T16:28:32.021230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:55.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:05.433386+0000","last_change":"2024-04-25T16:28:05.433416+0000","last_active":"2024-04-25T16:28:05.433386+0000","last_peered":"2024-04-25T16:28:05.433386+0000","last_clean":"2024-04-25T16:28:05.433386+0000","last_became_active":"2024-04-25T16:26:55.551556+0000","last_became_peered":"2024-04-25T16:26:55.551556+0000","last_unstale":"2024-04-25T16:28:05.433386+0000","last_undegraded":"2024-04-25T16:28:05.433386+0000","last_fullsized":"2024-04-25T16:28:05.433386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-25T16:28:05.433298+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-25T16:28:05.433298+0000","last_clean_scrub_stamp":"2024-04-25T16:28:05.433298+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:35.108855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:11.460612+0000","last_change":"2024-04-25T16:28:11.460659+0000","last_active":"2024-04-25T16:28:11.460612+0000","last_peered":"2024-04-25T16:28:11.460612+0000","last_clean":"2024-04-25T16:28:11.460612+0000","last_became_active":"2024-04-25T16:27:00.585966+0000","last_became_peered":"2024-04-25T16:27:00.585966+0000","last_unstale":"2024-04-25T16:28:11.460612+0000","last_undegraded":"2024-04-25T16:28:11.460612+0000","last_fullsized":"2024-04-25T16:28:11.460612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:11.460488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:11.460488+0000","last_clean_scrub_stamp":"2024-04-25T16:28:11.460488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:00.984034+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:41.153720+0000","last_change":"2024-04-25T16:28:41.153752+0000","last_active":"2024-04-25T16:28:41.153720+0000","last_peered":"2024-04-25T16:28:41.153720+0000","last_clean":"2024-04-25T16:28:41.153720+0000","last_became_active":"2024-04-25T16:26:56.568766+0000","last_became_peered":"2024-04-25T16:26:56.568766+0000","last_unstale":"2024-04-25T16:28:41.153720+0000","last_undegraded":"2024-04-25T16:28:41.153720+0000","last_fullsized":"2024-04-25T16:28:41.153720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:41.153623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:41.153623+0000","last_clean_scrub_stamp":"2024-04-25T16:28:41.153623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.330235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:08.554492+0000","last_change":"2024-04-25T16:28:08.554528+0000","last_active":"2024-04-25T16:28:08.554492+0000","last_peered":"2024-04-25T16:28:08.554492+0000","last_clean":"2024-04-25T16:28:08.554492+0000","last_became_active":"2024-04-25T16:26:55.551056+0000","last_became_peered":"2024-04-25T16:26:55.551056+0000","last_unstale":"2024-04-25T16:28:08.554492+0000","last_undegraded":"2024-04-25T16:28:08.554492+0000","last_fullsized":"2024-04-25T16:28:08.554492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-25T16:28:08.554406+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-25T16:28:08.554406+0000","last_clean_scrub_stamp":"2024-04-25T16:28:08.554406+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:53.392341+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:39.070733+0000","last_change":"2024-04-25T16:28:39.070768+0000","last_active":"2024-04-25T16:28:39.070733+0000","last_peered":"2024-04-25T16:28:39.070733+0000","last_clean":"2024-04-25T16:28:39.070733+0000","last_became_active":"2024-04-25T16:27:00.604741+0000","last_became_peered":"2024-04-25T16:27:00.604741+0000","last_unstale":"2024-04-25T16:28:39.070733+0000","last_undegraded":"2024-04-25T16:28:39.070733+0000","last_fullsized":"2024-04-25T16:28:39.070733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:39.070626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:39.070626+0000","last_clean_scrub_stamp":"2024-04-25T16:28:39.070626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:24.098809+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:42.131834+0000","last_change":"2024-04-25T16:28:42.131885+0000","last_active":"2024-04-25T16:28:42.131834+0000","last_peered":"2024-04-25T16:28:42.131834+0000","last_clean":"2024-04-25T16:28:42.131834+0000","last_became_active":"2024-04-25T16:26:56.568890+0000","last_became_peered":"2024-04-25T16:26:56.568890+0000","last_unstale":"2024-04-25T16:28:42.131834+0000","last_undegraded":"2024-04-25T16:28:42.131834+0000","last_fullsized":"2024-04-25T16:28:42.131834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:42.131709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:42.131709+0000","last_clean_scrub_stamp":"2024-04-25T16:28:42.131709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:14.436618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.808515+0000","last_change":"2024-04-25T16:28:17.808562+0000","last_active":"2024-04-25T16:28:17.808515+0000","last_peered":"2024-04-25T16:28:17.808515+0000","last_clean":"2024-04-25T16:28:17.808515+0000","last_became_active":"2024-04-25T16:26:55.552772+0000","last_became_peered":"2024-04-25T16:26:55.552772+0000","last_unstale":"2024-04-25T16:28:17.808515+0000","last_undegraded":"2024-04-25T16:28:17.808515+0000","last_fullsized":"2024-04-25T16:28:17.808515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-25T16:28:17.808422+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-25T16:28:17.808422+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.808422+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:25.703132+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:28.816377+0000","last_change":"2024-04-25T16:28:28.816404+0000","last_active":"2024-04-25T16:28:28.816377+0000","last_peered":"2024-04-25T16:28:28.816377+0000","last_clean":"2024-04-25T16:28:28.816377+0000","last_became_active":"2024-04-25T16:27:00.589314+0000","last_became_peered":"2024-04-25T16:27:00.589314+0000","last_unstale":"2024-04-25T16:28:28.816377+0000","last_undegraded":"2024-04-25T16:28:28.816377+0000","last_fullsized":"2024-04-25T16:28:28.816377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:28.816257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:28.816257+0000","last_clean_scrub_stamp":"2024-04-25T16:28:28.816257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:40.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:43.165103+0000","last_change":"2024-04-25T16:28:43.165129+0000","last_active":"2024-04-25T16:28:43.165103+0000","last_peered":"2024-04-25T16:28:43.165103+0000","last_clean":"2024-04-25T16:28:43.165103+0000","last_became_active":"2024-04-25T16:26:56.568704+0000","last_became_peered":"2024-04-25T16:26:56.568704+0000","last_unstale":"2024-04-25T16:28:43.165103+0000","last_undegraded":"2024-04-25T16:28:43.165103+0000","last_fullsized":"2024-04-25T16:28:43.165103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:43.165003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:43.165003+0000","last_clean_scrub_stamp":"2024-04-25T16:28:43.165003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:48.523167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'12","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:09.521548+0000","last_change":"2024-04-25T16:28:09.521591+0000","last_active":"2024-04-25T16:28:09.521548+0000","last_peered":"2024-04-25T16:28:09.521548+0000","last_clean":"2024-04-25T16:28:09.521548+0000","last_became_active":"2024-04-25T16:26:55.551787+0000","last_became_peered":"2024-04-25T16:26:55.551787+0000","last_unstale":"2024-04-25T16:28:09.521548+0000","last_undegraded":"2024-04-25T16:28:09.521548+0000","last_fullsized":"2024-04-25T16:28:09.521548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-25T16:28:09.521455+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-25T16:28:09.521455+0000","last_clean_scrub_stamp":"2024-04-25T16:28:09.521455+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:31.495884+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.493514+0000","last_change":"2024-04-25T16:28:12.493559+0000","last_active":"2024-04-25T16:28:12.493514+0000","last_peered":"2024-04-25T16:28:12.493514+0000","last_clean":"2024-04-25T16:28:12.493514+0000","last_became_active":"2024-04-25T16:27:00.587209+0000","last_became_peered":"2024-04-25T16:27:00.587209+0000","last_unstale":"2024-04-25T16:28:12.493514+0000","last_undegraded":"2024-04-25T16:28:12.493514+0000","last_fullsized":"2024-04-25T16:28:12.493514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.493394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.493394+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.493394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:37.500938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:33.178882+0000","last_change":"2024-04-25T16:28:33.178909+0000","last_active":"2024-04-25T16:28:33.178882+0000","last_peered":"2024-04-25T16:28:33.178882+0000","last_clean":"2024-04-25T16:28:33.178882+0000","last_became_active":"2024-04-25T16:26:56.554745+0000","last_became_peered":"2024-04-25T16:26:56.554745+0000","last_unstale":"2024-04-25T16:28:33.178882+0000","last_undegraded":"2024-04-25T16:28:33.178882+0000","last_fullsized":"2024-04-25T16:28:33.178882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:33.178788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:33.178788+0000","last_clean_scrub_stamp":"2024-04-25T16:28:33.178788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:10.714549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:28.144139+0000","last_change":"2024-04-25T16:28:28.144172+0000","last_active":"2024-04-25T16:28:28.144139+0000","last_peered":"2024-04-25T16:28:28.144139+0000","last_clean":"2024-04-25T16:28:28.144139+0000","last_became_active":"2024-04-25T16:26:55.551918+0000","last_became_peered":"2024-04-25T16:26:55.551918+0000","last_unstale":"2024-04-25T16:28:28.144139+0000","last_undegraded":"2024-04-25T16:28:28.144139+0000","last_fullsized":"2024-04-25T16:28:28.144139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:28.144045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:28.144045+0000","last_clean_scrub_stamp":"2024-04-25T16:28:28.144045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.339354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:30.700301+0000","last_change":"2024-04-25T16:28:30.700329+0000","last_active":"2024-04-25T16:28:30.700301+0000","last_peered":"2024-04-25T16:28:30.700301+0000","last_clean":"2024-04-25T16:28:30.700301+0000","last_became_active":"2024-04-25T16:27:00.604819+0000","last_became_peered":"2024-04-25T16:27:00.604819+0000","last_unstale":"2024-04-25T16:28:30.700301+0000","last_undegraded":"2024-04-25T16:28:30.700301+0000","last_fullsized":"2024-04-25T16:28:30.700301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:30.700214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:30.700214+0000","last_clean_scrub_stamp":"2024-04-25T16:28:30.700214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.395580+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:44.183838+0000","last_change":"2024-04-25T16:28:44.183868+0000","last_active":"2024-04-25T16:28:44.183838+0000","last_peered":"2024-04-25T16:28:44.183838+0000","last_clean":"2024-04-25T16:28:44.183838+0000","last_became_active":"2024-04-25T16:26:56.570146+0000","last_became_peered":"2024-04-25T16:26:56.570146+0000","last_unstale":"2024-04-25T16:28:44.183838+0000","last_undegraded":"2024-04-25T16:28:44.183838+0000","last_fullsized":"2024-04-25T16:28:44.183838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:44.183735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:44.183735+0000","last_clean_scrub_stamp":"2024-04-25T16:28:44.183735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.660442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.989783+0000","last_change":"2024-04-25T16:28:26.989830+0000","last_active":"2024-04-25T16:28:26.989783+0000","last_peered":"2024-04-25T16:28:26.989783+0000","last_clean":"2024-04-25T16:28:26.989783+0000","last_became_active":"2024-04-25T16:26:55.558643+0000","last_became_peered":"2024-04-25T16:26:55.558643+0000","last_unstale":"2024-04-25T16:28:26.989783+0000","last_undegraded":"2024-04-25T16:28:26.989783+0000","last_fullsized":"2024-04-25T16:28:26.989783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.989652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.989652+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.989652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:59.374695+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:29.675017+0000","last_change":"2024-04-25T16:28:29.675079+0000","last_active":"2024-04-25T16:28:29.675017+0000","last_peered":"2024-04-25T16:28:29.675017+0000","last_clean":"2024-04-25T16:28:29.675017+0000","last_became_active":"2024-04-25T16:27:00.602217+0000","last_became_peered":"2024-04-25T16:27:00.602217+0000","last_unstale":"2024-04-25T16:28:29.675017+0000","last_undegraded":"2024-04-25T16:28:29.675017+0000","last_fullsized":"2024-04-25T16:28:29.675017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:29.674863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:29.674863+0000","last_clean_scrub_stamp":"2024-04-25T16:28:29.674863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:39.438931+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:08.527534+0000","last_change":"2024-04-25T16:28:08.527555+0000","last_active":"2024-04-25T16:28:08.527534+0000","last_peered":"2024-04-25T16:28:08.527534+0000","last_clean":"2024-04-25T16:28:08.527534+0000","last_became_active":"2024-04-25T16:26:56.557754+0000","last_became_peered":"2024-04-25T16:26:56.557754+0000","last_unstale":"2024-04-25T16:28:08.527534+0000","last_undegraded":"2024-04-25T16:28:08.527534+0000","last_fullsized":"2024-04-25T16:28:08.527534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:08.527460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:08.527460+0000","last_clean_scrub_stamp":"2024-04-25T16:28:08.527460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:46.063221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'4","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:00.481192+0000","last_change":"2024-04-25T16:28:00.481248+0000","last_active":"2024-04-25T16:28:00.481192+0000","last_peered":"2024-04-25T16:28:00.481192+0000","last_clean":"2024-04-25T16:28:00.481192+0000","last_became_active":"2024-04-25T16:26:55.547340+0000","last_became_peered":"2024-04-25T16:26:55.547340+0000","last_unstale":"2024-04-25T16:28:00.481192+0000","last_undegraded":"2024-04-25T16:28:00.481192+0000","last_fullsized":"2024-04-25T16:28:00.481192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-25T16:28:00.481039+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-25T16:28:00.481039+0000","last_clean_scrub_stamp":"2024-04-25T16:28:00.481039+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:08.375749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:38.037332+0000","last_change":"2024-04-25T16:28:38.037383+0000","last_active":"2024-04-25T16:28:38.037332+0000","last_peered":"2024-04-25T16:28:38.037332+0000","last_clean":"2024-04-25T16:28:38.037332+0000","last_became_active":"2024-04-25T16:27:00.604693+0000","last_became_peered":"2024-04-25T16:27:00.604693+0000","last_unstale":"2024-04-25T16:28:38.037332+0000","last_undegraded":"2024-04-25T16:28:38.037332+0000","last_fullsized":"2024-04-25T16:28:38.037332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:38.037185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:38.037185+0000","last_clean_scrub_stamp":"2024-04-25T16:28:38.037185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:36.001618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:40.198098+0000","last_change":"2024-04-25T16:28:40.198127+0000","last_active":"2024-04-25T16:28:40.198098+0000","last_peered":"2024-04-25T16:28:40.198098+0000","last_clean":"2024-04-25T16:28:40.198098+0000","last_became_active":"2024-04-25T16:26:56.568026+0000","last_became_peered":"2024-04-25T16:26:56.568026+0000","last_unstale":"2024-04-25T16:28:40.198098+0000","last_undegraded":"2024-04-25T16:28:40.198098+0000","last_fullsized":"2024-04-25T16:28:40.198098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:40.198000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:40.198000+0000","last_clean_scrub_stamp":"2024-04-25T16:28:40.198000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:51.179303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'28","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.134875+0000","last_change":"2024-04-25T16:28:27.134922+0000","last_active":"2024-04-25T16:28:27.134875+0000","last_peered":"2024-04-25T16:28:27.134875+0000","last_clean":"2024-04-25T16:28:27.134875+0000","last_became_active":"2024-04-25T16:26:55.552550+0000","last_became_peered":"2024-04-25T16:26:55.552550+0000","last_unstale":"2024-04-25T16:28:27.134875+0000","last_undegraded":"2024-04-25T16:28:27.134875+0000","last_fullsized":"2024-04-25T16:28:27.134875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-25T16:28:27.134775+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-25T16:28:27.134775+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.134775+0000","objects_scrubbed":4,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:39.372904+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.359240+0000","last_change":"2024-04-25T16:28:15.359298+0000","last_active":"2024-04-25T16:28:15.359240+0000","last_peered":"2024-04-25T16:28:15.359240+0000","last_clean":"2024-04-25T16:28:15.359240+0000","last_became_active":"2024-04-25T16:27:00.583894+0000","last_became_peered":"2024-04-25T16:27:00.583894+0000","last_unstale":"2024-04-25T16:28:15.359240+0000","last_undegraded":"2024-04-25T16:28:15.359240+0000","last_fullsized":"2024-04-25T16:28:15.359240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.359100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.359100+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.359100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:27.480044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.458383+0000","last_change":"2024-04-25T16:28:15.458425+0000","last_active":"2024-04-25T16:28:15.458383+0000","last_peered":"2024-04-25T16:28:15.458383+0000","last_clean":"2024-04-25T16:28:15.458383+0000","last_became_active":"2024-04-25T16:27:36.700105+0000","last_became_peered":"2024-04-25T16:27:36.700105+0000","last_unstale":"2024-04-25T16:28:15.458383+0000","last_undegraded":"2024-04-25T16:28:15.458383+0000","last_fullsized":"2024-04-25T16:28:15.458383+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.458267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.458267+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.458267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:33.595091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:08.430920+0000","last_change":"2024-04-25T16:28:08.430951+0000","last_active":"2024-04-25T16:28:08.430920+0000","last_peered":"2024-04-25T16:28:08.430920+0000","last_clean":"2024-04-25T16:28:08.430920+0000","last_became_active":"2024-04-25T16:26:55.558755+0000","last_became_peered":"2024-04-25T16:26:55.558755+0000","last_unstale":"2024-04-25T16:28:08.430920+0000","last_undegraded":"2024-04-25T16:28:08.430920+0000","last_fullsized":"2024-04-25T16:28:08.430920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-25T16:28:08.430845+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-25T16:28:08.430845+0000","last_clean_scrub_stamp":"2024-04-25T16:28:08.430845+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:10.304775+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.773109+0000","last_change":"2024-04-25T16:28:27.773162+0000","last_active":"2024-04-25T16:28:27.773109+0000","last_peered":"2024-04-25T16:28:27.773109+0000","last_clean":"2024-04-25T16:28:27.773109+0000","last_became_active":"2024-04-25T16:27:00.588163+0000","last_became_peered":"2024-04-25T16:27:00.588163+0000","last_unstale":"2024-04-25T16:28:27.773109+0000","last_undegraded":"2024-04-25T16:28:27.773109+0000","last_fullsized":"2024-04-25T16:28:27.773109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:27.772993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:27.772993+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.772993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:00.158036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:09.489587+0000","last_change":"2024-04-25T16:28:09.489610+0000","last_active":"2024-04-25T16:28:09.489587+0000","last_peered":"2024-04-25T16:28:09.489587+0000","last_clean":"2024-04-25T16:28:09.489587+0000","last_became_active":"2024-04-25T16:26:56.557593+0000","last_became_peered":"2024-04-25T16:26:56.557593+0000","last_unstale":"2024-04-25T16:28:09.489587+0000","last_undegraded":"2024-04-25T16:28:09.489587+0000","last_fullsized":"2024-04-25T16:28:09.489587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:09.489480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:09.489480+0000","last_clean_scrub_stamp":"2024-04-25T16:28:09.489480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:41.874523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.969428+0000","last_change":"2024-04-25T16:28:27.969475+0000","last_active":"2024-04-25T16:28:27.969428+0000","last_peered":"2024-04-25T16:28:27.969428+0000","last_clean":"2024-04-25T16:28:27.969428+0000","last_became_active":"2024-04-25T16:26:55.558607+0000","last_became_peered":"2024-04-25T16:26:55.558607+0000","last_unstale":"2024-04-25T16:28:27.969428+0000","last_undegraded":"2024-04-25T16:28:27.969428+0000","last_fullsized":"2024-04-25T16:28:27.969428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:27.969287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:27.969287+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.969287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:40.090231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:27.726768+0000","last_change":"2024-04-25T16:28:27.726797+0000","last_active":"2024-04-25T16:28:27.726768+0000","last_peered":"2024-04-25T16:28:27.726768+0000","last_clean":"2024-04-25T16:28:27.726768+0000","last_became_active":"2024-04-25T16:27:00.584665+0000","last_became_peered":"2024-04-25T16:27:00.584665+0000","last_unstale":"2024-04-25T16:28:27.726768+0000","last_undegraded":"2024-04-25T16:28:27.726768+0000","last_fullsized":"2024-04-25T16:28:27.726768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-25T16:28:27.726687+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-25T16:28:27.726687+0000","last_clean_scrub_stamp":"2024-04-25T16:28:27.726687+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:13.150019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:34.990210+0000","last_change":"2024-04-25T16:28:34.990238+0000","last_active":"2024-04-25T16:28:34.990210+0000","last_peered":"2024-04-25T16:28:34.990210+0000","last_clean":"2024-04-25T16:28:34.990210+0000","last_became_active":"2024-04-25T16:26:56.572009+0000","last_became_peered":"2024-04-25T16:26:56.572009+0000","last_unstale":"2024-04-25T16:28:34.990210+0000","last_undegraded":"2024-04-25T16:28:34.990210+0000","last_fullsized":"2024-04-25T16:28:34.990210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:34.990110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:34.990110+0000","last_clean_scrub_stamp":"2024-04-25T16:28:34.990110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:07.295019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:30.175616+0000","last_change":"2024-04-25T16:28:30.175643+0000","last_active":"2024-04-25T16:28:30.175616+0000","last_peered":"2024-04-25T16:28:30.175616+0000","last_clean":"2024-04-25T16:28:30.175616+0000","last_became_active":"2024-04-25T16:26:55.559244+0000","last_became_peered":"2024-04-25T16:26:55.559244+0000","last_unstale":"2024-04-25T16:28:30.175616+0000","last_undegraded":"2024-04-25T16:28:30.175616+0000","last_fullsized":"2024-04-25T16:28:30.175616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:30.175518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:30.175518+0000","last_clean_scrub_stamp":"2024-04-25T16:28:30.175518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:18.154014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:32.658770+0000","last_change":"2024-04-25T16:28:32.658824+0000","last_active":"2024-04-25T16:28:32.658770+0000","last_peered":"2024-04-25T16:28:32.658770+0000","last_clean":"2024-04-25T16:28:32.658770+0000","last_became_active":"2024-04-25T16:27:00.607374+0000","last_became_peered":"2024-04-25T16:27:00.607374+0000","last_unstale":"2024-04-25T16:28:32.658770+0000","last_undegraded":"2024-04-25T16:28:32.658770+0000","last_fullsized":"2024-04-25T16:28:32.658770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:32.658623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:32.658623+0000","last_clean_scrub_stamp":"2024-04-25T16:28:32.658623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:55.425145+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:15.532576+0000","last_change":"2024-04-25T16:28:15.532607+0000","last_active":"2024-04-25T16:28:15.532576+0000","last_peered":"2024-04-25T16:28:15.532576+0000","last_clean":"2024-04-25T16:28:15.532576+0000","last_became_active":"2024-04-25T16:27:36.702077+0000","last_became_peered":"2024-04-25T16:27:36.702077+0000","last_unstale":"2024-04-25T16:28:15.532576+0000","last_undegraded":"2024-04-25T16:28:15.532576+0000","last_fullsized":"2024-04-25T16:28:15.532576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:15.532484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:15.532484+0000","last_clean_scrub_stamp":"2024-04-25T16:28:15.532484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:26.513787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,2],"acting":[0,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:19.411015+0000","last_change":"2024-04-25T16:28:19.411051+0000","last_active":"2024-04-25T16:28:19.411015+0000","last_peered":"2024-04-25T16:28:19.411015+0000","last_clean":"2024-04-25T16:28:19.411015+0000","last_became_active":"2024-04-25T16:27:36.701427+0000","last_became_peered":"2024-04-25T16:27:36.701427+0000","last_unstale":"2024-04-25T16:28:19.411015+0000","last_undegraded":"2024-04-25T16:28:19.411015+0000","last_fullsized":"2024-04-25T16:28:19.411015+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:19.410911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:19.410911+0000","last_clean_scrub_stamp":"2024-04-25T16:28:19.410911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:42.643237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.018943+0000","last_change":"2024-04-25T16:28:26.018974+0000","last_active":"2024-04-25T16:28:26.018943+0000","last_peered":"2024-04-25T16:28:26.018943+0000","last_clean":"2024-04-25T16:28:26.018943+0000","last_became_active":"2024-04-25T16:26:55.555981+0000","last_became_peered":"2024-04-25T16:26:55.555981+0000","last_unstale":"2024-04-25T16:28:26.018943+0000","last_undegraded":"2024-04-25T16:28:26.018943+0000","last_fullsized":"2024-04-25T16:28:26.018943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.018830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.018830+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.018830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:01.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:40.222928+0000","last_change":"2024-04-25T16:28:40.222960+0000","last_active":"2024-04-25T16:28:40.222928+0000","last_peered":"2024-04-25T16:28:40.222928+0000","last_clean":"2024-04-25T16:28:40.222928+0000","last_became_active":"2024-04-25T16:27:36.765669+0000","last_became_peered":"2024-04-25T16:27:36.765669+0000","last_unstale":"2024-04-25T16:28:40.222928+0000","last_undegraded":"2024-04-25T16:28:40.222928+0000","last_fullsized":"2024-04-25T16:28:40.222928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:40.222833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:40.222833+0000","last_clean_scrub_stamp":"2024-04-25T16:28:40.222833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:51.204135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,0],"acting":[6,5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:21.676037+0000","last_change":"2024-04-25T16:28:21.676087+0000","last_active":"2024-04-25T16:28:21.676037+0000","last_peered":"2024-04-25T16:28:21.676037+0000","last_clean":"2024-04-25T16:28:21.676037+0000","last_became_active":"2024-04-25T16:26:56.550446+0000","last_became_peered":"2024-04-25T16:26:56.550446+0000","last_unstale":"2024-04-25T16:28:21.676037+0000","last_undegraded":"2024-04-25T16:28:21.676037+0000","last_fullsized":"2024-04-25T16:28:21.676037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:21.675943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:21.675943+0000","last_clean_scrub_stamp":"2024-04-25T16:28:21.675943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:46.683488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:17.750756+0000","last_change":"2024-04-25T16:28:17.750781+0000","last_active":"2024-04-25T16:28:17.750756+0000","last_peered":"2024-04-25T16:28:17.750756+0000","last_clean":"2024-04-25T16:28:17.750756+0000","last_became_active":"2024-04-25T16:26:55.552340+0000","last_became_peered":"2024-04-25T16:26:55.552340+0000","last_unstale":"2024-04-25T16:28:17.750756+0000","last_undegraded":"2024-04-25T16:28:17.750756+0000","last_fullsized":"2024-04-25T16:28:17.750756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:17.750678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:17.750678+0000","last_clean_scrub_stamp":"2024-04-25T16:28:17.750678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.286439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.158150+0000","last_change":"2024-04-25T16:28:26.158182+0000","last_active":"2024-04-25T16:28:26.158150+0000","last_peered":"2024-04-25T16:28:26.158150+0000","last_clean":"2024-04-25T16:28:26.158150+0000","last_became_active":"2024-04-25T16:26:55.551407+0000","last_became_peered":"2024-04-25T16:26:55.551407+0000","last_unstale":"2024-04-25T16:28:26.158150+0000","last_undegraded":"2024-04-25T16:28:26.158150+0000","last_fullsized":"2024-04-25T16:28:26.158150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T16:28:26.158055+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T16:28:26.158055+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.158055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:28.239379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:07.527654+0000","last_change":"2024-04-25T16:28:07.527683+0000","last_active":"2024-04-25T16:28:07.527654+0000","last_peered":"2024-04-25T16:28:07.527654+0000","last_clean":"2024-04-25T16:28:07.527654+0000","last_became_active":"2024-04-25T16:26:56.555909+0000","last_became_peered":"2024-04-25T16:26:56.555909+0000","last_unstale":"2024-04-25T16:28:07.527654+0000","last_undegraded":"2024-04-25T16:28:07.527654+0000","last_fullsized":"2024-04-25T16:28:07.527654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:07.527553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:07.527553+0000","last_clean_scrub_stamp":"2024-04-25T16:28:07.527553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:26.746231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.456014+0000","last_change":"2024-04-25T16:28:14.456042+0000","last_active":"2024-04-25T16:28:14.456014+0000","last_peered":"2024-04-25T16:28:14.456014+0000","last_clean":"2024-04-25T16:28:14.456014+0000","last_became_active":"2024-04-25T16:27:00.587773+0000","last_became_peered":"2024-04-25T16:27:00.587773+0000","last_unstale":"2024-04-25T16:28:14.456014+0000","last_undegraded":"2024-04-25T16:28:14.456014+0000","last_fullsized":"2024-04-25T16:28:14.456014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.455918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.455918+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.455918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:32.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:19.719920+0000","last_change":"2024-04-25T16:28:19.719967+0000","last_active":"2024-04-25T16:28:19.719920+0000","last_peered":"2024-04-25T16:28:19.719920+0000","last_clean":"2024-04-25T16:28:19.719920+0000","last_became_active":"2024-04-25T16:26:55.558757+0000","last_became_peered":"2024-04-25T16:26:55.558757+0000","last_unstale":"2024-04-25T16:28:19.719920+0000","last_undegraded":"2024-04-25T16:28:19.719920+0000","last_fullsized":"2024-04-25T16:28:19.719920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T16:28:19.719762+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T16:28:19.719762+0000","last_clean_scrub_stamp":"2024-04-25T16:28:19.719762+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:34.965212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:19.861440+0000","last_change":"2024-04-25T16:28:19.861473+0000","last_active":"2024-04-25T16:28:19.861440+0000","last_peered":"2024-04-25T16:28:19.861440+0000","last_clean":"2024-04-25T16:28:19.861440+0000","last_became_active":"2024-04-25T16:26:56.552563+0000","last_became_peered":"2024-04-25T16:26:56.552563+0000","last_unstale":"2024-04-25T16:28:19.861440+0000","last_undegraded":"2024-04-25T16:28:19.861440+0000","last_fullsized":"2024-04-25T16:28:19.861440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:19.861334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:19.861334+0000","last_clean_scrub_stamp":"2024-04-25T16:28:19.861334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:21.942657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:26.689327+0000","last_change":"2024-04-25T16:28:26.689356+0000","last_active":"2024-04-25T16:28:26.689327+0000","last_peered":"2024-04-25T16:28:26.689327+0000","last_clean":"2024-04-25T16:28:26.689327+0000","last_became_active":"2024-04-25T16:27:00.583635+0000","last_became_peered":"2024-04-25T16:27:00.583635+0000","last_unstale":"2024-04-25T16:28:26.689327+0000","last_undegraded":"2024-04-25T16:28:26.689327+0000","last_fullsized":"2024-04-25T16:28:26.689327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:26.689225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:26.689225+0000","last_clean_scrub_stamp":"2024-04-25T16:28:26.689225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:04.033610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:32.215965+0000","last_change":"2024-04-25T16:28:32.215992+0000","last_active":"2024-04-25T16:28:32.215965+0000","last_peered":"2024-04-25T16:28:32.215965+0000","last_clean":"2024-04-25T16:28:32.215965+0000","last_became_active":"2024-04-25T16:26:56.555183+0000","last_became_peered":"2024-04-25T16:26:56.555183+0000","last_unstale":"2024-04-25T16:28:32.215965+0000","last_undegraded":"2024-04-25T16:28:32.215965+0000","last_fullsized":"2024-04-25T16:28:32.215965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:32.215838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:32.215838+0000","last_clean_scrub_stamp":"2024-04-25T16:28:32.215838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:51.434517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.449230+0000","last_change":"2024-04-25T16:28:13.449257+0000","last_active":"2024-04-25T16:28:13.449230+0000","last_peered":"2024-04-25T16:28:13.449230+0000","last_clean":"2024-04-25T16:28:13.449230+0000","last_became_active":"2024-04-25T16:26:56.556506+0000","last_became_peered":"2024-04-25T16:26:56.556506+0000","last_unstale":"2024-04-25T16:28:13.449230+0000","last_undegraded":"2024-04-25T16:28:13.449230+0000","last_fullsized":"2024-04-25T16:28:13.449230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.449148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.449148+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.449148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:13.025915+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:16.427112+0000","last_change":"2024-04-25T16:28:16.427156+0000","last_active":"2024-04-25T16:28:16.427112+0000","last_peered":"2024-04-25T16:28:16.427112+0000","last_clean":"2024-04-25T16:28:16.427112+0000","last_became_active":"2024-04-25T16:27:36.701156+0000","last_became_peered":"2024-04-25T16:27:36.701156+0000","last_unstale":"2024-04-25T16:28:16.427112+0000","last_undegraded":"2024-04-25T16:28:16.427112+0000","last_fullsized":"2024-04-25T16:28:16.427112+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:16.426964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:16.426964+0000","last_clean_scrub_stamp":"2024-04-25T16:28:16.426964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:19.917910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:05.547570+0000","last_change":"2024-04-25T16:28:05.547614+0000","last_active":"2024-04-25T16:28:05.547570+0000","last_peered":"2024-04-25T16:28:05.547570+0000","last_clean":"2024-04-25T16:28:05.547570+0000","last_became_active":"2024-04-25T16:26:56.554376+0000","last_became_peered":"2024-04-25T16:26:56.554376+0000","last_unstale":"2024-04-25T16:28:05.547570+0000","last_undegraded":"2024-04-25T16:28:05.547570+0000","last_fullsized":"2024-04-25T16:28:05.547570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:05.547456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:05.547456+0000","last_clean_scrub_stamp":"2024-04-25T16:28:05.547456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:38.600732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:24.668238+0000","last_change":"2024-04-25T16:28:24.668282+0000","last_active":"2024-04-25T16:28:24.668238+0000","last_peered":"2024-04-25T16:28:24.668238+0000","last_clean":"2024-04-25T16:28:24.668238+0000","last_became_active":"2024-04-25T16:26:56.556038+0000","last_became_peered":"2024-04-25T16:26:56.556038+0000","last_unstale":"2024-04-25T16:28:24.668238+0000","last_undegraded":"2024-04-25T16:28:24.668238+0000","last_fullsized":"2024-04-25T16:28:24.668238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:24.668106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:24.668106+0000","last_clean_scrub_stamp":"2024-04-25T16:28:24.668106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:55.247900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.301436+0000","last_change":"2024-04-25T16:28:13.301477+0000","last_active":"2024-04-25T16:28:13.301436+0000","last_peered":"2024-04-25T16:28:13.301436+0000","last_clean":"2024-04-25T16:28:13.301436+0000","last_became_active":"2024-04-25T16:26:56.552004+0000","last_became_peered":"2024-04-25T16:26:56.552004+0000","last_unstale":"2024-04-25T16:28:13.301436+0000","last_undegraded":"2024-04-25T16:28:13.301436+0000","last_fullsized":"2024-04-25T16:28:13.301436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.301298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.301298+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.301298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:50.837059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:35.214952+0000","last_change":"2024-04-25T16:28:35.214983+0000","last_active":"2024-04-25T16:28:35.214952+0000","last_peered":"2024-04-25T16:28:35.214952+0000","last_clean":"2024-04-25T16:28:35.214952+0000","last_became_active":"2024-04-25T16:26:56.557573+0000","last_became_peered":"2024-04-25T16:26:56.557573+0000","last_unstale":"2024-04-25T16:28:35.214952+0000","last_undegraded":"2024-04-25T16:28:35.214952+0000","last_fullsized":"2024-04-25T16:28:35.214952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:35.214860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:35.214860+0000","last_clean_scrub_stamp":"2024-04-25T16:28:35.214860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:47.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:22.696689+0000","last_change":"2024-04-25T16:28:22.696720+0000","last_active":"2024-04-25T16:28:22.696689+0000","last_peered":"2024-04-25T16:28:22.696689+0000","last_clean":"2024-04-25T16:28:22.696689+0000","last_became_active":"2024-04-25T16:26:56.553095+0000","last_became_peered":"2024-04-25T16:26:56.553095+0000","last_unstale":"2024-04-25T16:28:22.696689+0000","last_undegraded":"2024-04-25T16:28:22.696689+0000","last_fullsized":"2024-04-25T16:28:22.696689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:22.696584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:22.696584+0000","last_clean_scrub_stamp":"2024-04-25T16:28:22.696584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.771045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:29.964894+0000","last_change":"2024-04-25T16:28:29.964959+0000","last_active":"2024-04-25T16:28:29.964894+0000","last_peered":"2024-04-25T16:28:29.964894+0000","last_clean":"2024-04-25T16:28:29.964894+0000","last_became_active":"2024-04-25T16:26:56.564052+0000","last_became_peered":"2024-04-25T16:26:56.564052+0000","last_unstale":"2024-04-25T16:28:29.964894+0000","last_undegraded":"2024-04-25T16:28:29.964894+0000","last_fullsized":"2024-04-25T16:28:29.964894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:29.964784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:29.964784+0000","last_clean_scrub_stamp":"2024-04-25T16:28:29.964784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:54.972328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:16.727278+0000","last_change":"2024-04-25T16:28:16.727319+0000","last_active":"2024-04-25T16:28:16.727278+0000","last_peered":"2024-04-25T16:28:16.727278+0000","last_clean":"2024-04-25T16:28:16.727278+0000","last_became_active":"2024-04-25T16:26:56.557190+0000","last_became_peered":"2024-04-25T16:26:56.557190+0000","last_unstale":"2024-04-25T16:28:16.727278+0000","last_undegraded":"2024-04-25T16:28:16.727278+0000","last_fullsized":"2024-04-25T16:28:16.727278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:16.727153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:16.727153+0000","last_clean_scrub_stamp":"2024-04-25T16:28:16.727153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:35.945831+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:36.020921+0000","last_change":"2024-04-25T16:28:36.020951+0000","last_active":"2024-04-25T16:28:36.020921+0000","last_peered":"2024-04-25T16:28:36.020921+0000","last_clean":"2024-04-25T16:28:36.020921+0000","last_became_active":"2024-04-25T16:26:56.572172+0000","last_became_peered":"2024-04-25T16:26:56.572172+0000","last_unstale":"2024-04-25T16:28:36.020921+0000","last_undegraded":"2024-04-25T16:28:36.020921+0000","last_fullsized":"2024-04-25T16:28:36.020921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:36.020807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:36.020807+0000","last_clean_scrub_stamp":"2024-04-25T16:28:36.020807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:41.378971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:10.471272+0000","last_change":"2024-04-25T16:28:10.471321+0000","last_active":"2024-04-25T16:28:10.471272+0000","last_peered":"2024-04-25T16:28:10.471272+0000","last_clean":"2024-04-25T16:28:10.471272+0000","last_became_active":"2024-04-25T16:26:56.562641+0000","last_became_peered":"2024-04-25T16:26:56.562641+0000","last_unstale":"2024-04-25T16:28:10.471272+0000","last_undegraded":"2024-04-25T16:28:10.471272+0000","last_fullsized":"2024-04-25T16:28:10.471272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:10.471146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:10.471146+0000","last_clean_scrub_stamp":"2024-04-25T16:28:10.471146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:22.592090+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:34.186460+0000","last_change":"2024-04-25T16:28:34.186517+0000","last_active":"2024-04-25T16:28:34.186460+0000","last_peered":"2024-04-25T16:28:34.186460+0000","last_clean":"2024-04-25T16:28:34.186460+0000","last_became_active":"2024-04-25T16:26:56.555490+0000","last_became_peered":"2024-04-25T16:26:56.555490+0000","last_unstale":"2024-04-25T16:28:34.186460+0000","last_undegraded":"2024-04-25T16:28:34.186460+0000","last_fullsized":"2024-04-25T16:28:34.186460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:34.186353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:34.186353+0000","last_clean_scrub_stamp":"2024-04-25T16:28:34.186353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:21.452700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:45.218724+0000","last_change":"2024-04-25T16:28:45.218751+0000","last_active":"2024-04-25T16:28:45.218724+0000","last_peered":"2024-04-25T16:28:45.218724+0000","last_clean":"2024-04-25T16:28:45.218724+0000","last_became_active":"2024-04-25T16:26:56.568390+0000","last_became_peered":"2024-04-25T16:26:56.568390+0000","last_unstale":"2024-04-25T16:28:45.218724+0000","last_undegraded":"2024-04-25T16:28:45.218724+0000","last_fullsized":"2024-04-25T16:28:45.218724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:45.218588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:45.218588+0000","last_clean_scrub_stamp":"2024-04-25T16:28:45.218588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:43.183021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.493300+0000","last_change":"2024-04-25T16:28:12.493341+0000","last_active":"2024-04-25T16:28:12.493300+0000","last_peered":"2024-04-25T16:28:12.493300+0000","last_clean":"2024-04-25T16:28:12.493300+0000","last_became_active":"2024-04-25T16:26:56.555217+0000","last_became_peered":"2024-04-25T16:26:56.555217+0000","last_unstale":"2024-04-25T16:28:12.493300+0000","last_undegraded":"2024-04-25T16:28:12.493300+0000","last_fullsized":"2024-04-25T16:28:12.493300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.493175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.493175+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.493175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:08.943179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:36.990629+0000","last_change":"2024-04-25T16:28:36.990658+0000","last_active":"2024-04-25T16:28:36.990629+0000","last_peered":"2024-04-25T16:28:36.990629+0000","last_clean":"2024-04-25T16:28:36.990629+0000","last_became_active":"2024-04-25T16:26:56.572837+0000","last_became_peered":"2024-04-25T16:26:56.572837+0000","last_unstale":"2024-04-25T16:28:36.990629+0000","last_undegraded":"2024-04-25T16:28:36.990629+0000","last_fullsized":"2024-04-25T16:28:36.990629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:36.990534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:36.990534+0000","last_clean_scrub_stamp":"2024-04-25T16:28:36.990534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:52.467242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:34.024747+0000","last_change":"2024-04-25T16:28:34.024791+0000","last_active":"2024-04-25T16:28:34.024747+0000","last_peered":"2024-04-25T16:28:34.024747+0000","last_clean":"2024-04-25T16:28:34.024747+0000","last_became_active":"2024-04-25T16:26:56.566684+0000","last_became_peered":"2024-04-25T16:26:56.566684+0000","last_unstale":"2024-04-25T16:28:34.024747+0000","last_undegraded":"2024-04-25T16:28:34.024747+0000","last_fullsized":"2024-04-25T16:28:34.024747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:34.024597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:34.024597+0000","last_clean_scrub_stamp":"2024-04-25T16:28:34.024597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:12.201209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.339993+0000","last_change":"2024-04-25T16:28:14.340050+0000","last_active":"2024-04-25T16:28:14.339993+0000","last_peered":"2024-04-25T16:28:14.339993+0000","last_clean":"2024-04-25T16:28:14.339993+0000","last_became_active":"2024-04-25T16:26:56.558475+0000","last_became_peered":"2024-04-25T16:26:56.558475+0000","last_unstale":"2024-04-25T16:28:14.339993+0000","last_undegraded":"2024-04-25T16:28:14.339993+0000","last_fullsized":"2024-04-25T16:28:14.339993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.339872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.339872+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.339872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:46.724915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:03.574699+0000","last_change":"2024-04-25T16:28:03.574722+0000","last_active":"2024-04-25T16:28:03.574699+0000","last_peered":"2024-04-25T16:28:03.574699+0000","last_clean":"2024-04-25T16:28:03.574699+0000","last_became_active":"2024-04-25T16:26:56.550272+0000","last_became_peered":"2024-04-25T16:26:56.550272+0000","last_unstale":"2024-04-25T16:28:03.574699+0000","last_undegraded":"2024-04-25T16:28:03.574699+0000","last_fullsized":"2024-04-25T16:28:03.574699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:03.574620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:03.574620+0000","last_clean_scrub_stamp":"2024-04-25T16:28:03.574620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:30.769929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:14.723433+0000","last_change":"2024-04-25T16:28:14.723461+0000","last_active":"2024-04-25T16:28:14.723433+0000","last_peered":"2024-04-25T16:28:14.723433+0000","last_clean":"2024-04-25T16:28:14.723433+0000","last_became_active":"2024-04-25T16:26:56.562830+0000","last_became_peered":"2024-04-25T16:26:56.562830+0000","last_unstale":"2024-04-25T16:28:14.723433+0000","last_undegraded":"2024-04-25T16:28:14.723433+0000","last_fullsized":"2024-04-25T16:28:14.723433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:14.723299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:14.723299+0000","last_clean_scrub_stamp":"2024-04-25T16:28:14.723299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:47.776575+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:21.929950+0000","last_change":"2024-04-25T16:28:21.929981+0000","last_active":"2024-04-25T16:28:21.929950+0000","last_peered":"2024-04-25T16:28:21.929950+0000","last_clean":"2024-04-25T16:28:21.929950+0000","last_became_active":"2024-04-25T16:26:56.557749+0000","last_became_peered":"2024-04-25T16:26:56.557749+0000","last_unstale":"2024-04-25T16:28:21.929950+0000","last_undegraded":"2024-04-25T16:28:21.929950+0000","last_fullsized":"2024-04-25T16:28:21.929950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:21.929827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:21.929827+0000","last_clean_scrub_stamp":"2024-04-25T16:28:21.929827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:23.803757+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:37.192881+0000","last_change":"2024-04-25T16:28:37.192917+0000","last_active":"2024-04-25T16:28:37.192881+0000","last_peered":"2024-04-25T16:28:37.192881+0000","last_clean":"2024-04-25T16:28:37.192881+0000","last_became_active":"2024-04-25T16:26:56.567255+0000","last_became_peered":"2024-04-25T16:26:56.567255+0000","last_unstale":"2024-04-25T16:28:37.192881+0000","last_undegraded":"2024-04-25T16:28:37.192881+0000","last_fullsized":"2024-04-25T16:28:37.192881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:37.192785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:37.192785+0000","last_clean_scrub_stamp":"2024-04-25T16:28:37.192785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:44.853136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:36.207997+0000","last_change":"2024-04-25T16:28:36.208024+0000","last_active":"2024-04-25T16:28:36.207997+0000","last_peered":"2024-04-25T16:28:36.207997+0000","last_clean":"2024-04-25T16:28:36.207997+0000","last_became_active":"2024-04-25T16:26:56.563171+0000","last_became_peered":"2024-04-25T16:26:56.563171+0000","last_unstale":"2024-04-25T16:28:36.207997+0000","last_undegraded":"2024-04-25T16:28:36.207997+0000","last_fullsized":"2024-04-25T16:28:36.207997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:36.207888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:36.207888+0000","last_clean_scrub_stamp":"2024-04-25T16:28:36.207888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:25.731435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:47.305161+0000","last_change":"2024-04-25T16:28:47.305224+0000","last_active":"2024-04-25T16:28:47.305161+0000","last_peered":"2024-04-25T16:28:47.305161+0000","last_clean":"2024-04-25T16:28:47.305161+0000","last_became_active":"2024-04-25T16:27:36.703485+0000","last_became_peered":"2024-04-25T16:27:36.703485+0000","last_unstale":"2024-04-25T16:28:47.305161+0000","last_undegraded":"2024-04-25T16:28:47.305161+0000","last_fullsized":"2024-04-25T16:28:47.305161+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:47.305029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:47.305029+0000","last_clean_scrub_stamp":"2024-04-25T16:28:47.305029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:37.669078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:13.704765+0000","last_change":"2024-04-25T16:28:13.704807+0000","last_active":"2024-04-25T16:28:13.704765+0000","last_peered":"2024-04-25T16:28:13.704765+0000","last_clean":"2024-04-25T16:28:13.704765+0000","last_became_active":"2024-04-25T16:26:56.564169+0000","last_became_peered":"2024-04-25T16:26:56.564169+0000","last_unstale":"2024-04-25T16:28:13.704765+0000","last_undegraded":"2024-04-25T16:28:13.704765+0000","last_fullsized":"2024-04-25T16:28:13.704765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:13.704644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:13.704644+0000","last_clean_scrub_stamp":"2024-04-25T16:28:13.704644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:15.127800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:39.155689+0000","last_change":"2024-04-25T16:28:39.155734+0000","last_active":"2024-04-25T16:28:39.155689+0000","last_peered":"2024-04-25T16:28:39.155689+0000","last_clean":"2024-04-25T16:28:39.155689+0000","last_became_active":"2024-04-25T16:26:56.562535+0000","last_became_peered":"2024-04-25T16:26:56.562535+0000","last_unstale":"2024-04-25T16:28:39.155689+0000","last_undegraded":"2024-04-25T16:28:39.155689+0000","last_fullsized":"2024-04-25T16:28:39.155689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:39.155569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:39.155569+0000","last_clean_scrub_stamp":"2024-04-25T16:28:39.155569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.387895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:32.988021+0000","last_change":"2024-04-25T16:28:32.988047+0000","last_active":"2024-04-25T16:28:32.988021+0000","last_peered":"2024-04-25T16:28:32.988021+0000","last_clean":"2024-04-25T16:28:32.988021+0000","last_became_active":"2024-04-25T16:26:56.566551+0000","last_became_peered":"2024-04-25T16:26:56.566551+0000","last_unstale":"2024-04-25T16:28:32.988021+0000","last_undegraded":"2024-04-25T16:28:32.988021+0000","last_fullsized":"2024-04-25T16:28:32.988021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:32.987917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:32.987917+0000","last_clean_scrub_stamp":"2024-04-25T16:28:32.987917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:43.969220+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:30.189137+0000","last_change":"2024-04-25T16:28:30.189199+0000","last_active":"2024-04-25T16:28:30.189137+0000","last_peered":"2024-04-25T16:28:30.189137+0000","last_clean":"2024-04-25T16:28:30.189137+0000","last_became_active":"2024-04-25T16:26:56.567323+0000","last_became_peered":"2024-04-25T16:26:56.567323+0000","last_unstale":"2024-04-25T16:28:30.189137+0000","last_undegraded":"2024-04-25T16:28:30.189137+0000","last_fullsized":"2024-04-25T16:28:30.189137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:30.189017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:30.189017+0000","last_clean_scrub_stamp":"2024-04-25T16:28:30.189017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:57:03.242292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:22.083000+0000","last_change":"2024-04-25T16:28:22.083029+0000","last_active":"2024-04-25T16:28:22.083000+0000","last_peered":"2024-04-25T16:28:22.083000+0000","last_clean":"2024-04-25T16:28:22.083000+0000","last_became_active":"2024-04-25T16:26:56.564752+0000","last_became_peered":"2024-04-25T16:26:56.564752+0000","last_unstale":"2024-04-25T16:28:22.083000+0000","last_undegraded":"2024-04-25T16:28:22.083000+0000","last_fullsized":"2024-04-25T16:28:22.083000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:22.082903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:22.082903+0000","last_clean_scrub_stamp":"2024-04-25T16:28:22.082903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:49.278211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:02.525404+0000","last_change":"2024-04-25T16:28:02.525439+0000","last_active":"2024-04-25T16:28:02.525404+0000","last_peered":"2024-04-25T16:28:02.525404+0000","last_clean":"2024-04-25T16:28:02.525404+0000","last_became_active":"2024-04-25T16:26:56.546251+0000","last_became_peered":"2024-04-25T16:26:56.546251+0000","last_unstale":"2024-04-25T16:28:02.525404+0000","last_undegraded":"2024-04-25T16:28:02.525404+0000","last_fullsized":"2024-04-25T16:28:02.525404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:02.525282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:02.525282+0000","last_clean_scrub_stamp":"2024-04-25T16:28:02.525282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:14.763411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:12.309906+0000","last_change":"2024-04-25T16:28:12.309946+0000","last_active":"2024-04-25T16:28:12.309906+0000","last_peered":"2024-04-25T16:28:12.309906+0000","last_clean":"2024-04-25T16:28:12.309906+0000","last_became_active":"2024-04-25T16:26:56.548589+0000","last_became_peered":"2024-04-25T16:26:56.548589+0000","last_unstale":"2024-04-25T16:28:12.309906+0000","last_undegraded":"2024-04-25T16:28:12.309906+0000","last_fullsized":"2024-04-25T16:28:12.309906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:12.309787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:12.309787+0000","last_clean_scrub_stamp":"2024-04-25T16:28:12.309787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:42.695977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T16:28:31.160637+0000","last_change":"2024-04-25T16:28:31.160669+0000","last_active":"2024-04-25T16:28:31.160637+0000","last_peered":"2024-04-25T16:28:31.160637+0000","last_clean":"2024-04-25T16:28:31.160637+0000","last_became_active":"2024-04-25T16:26:56.564041+0000","last_became_peered":"2024-04-25T16:26:56.564041+0000","last_unstale":"2024-04-25T16:28:31.160637+0000","last_undegraded":"2024-04-25T16:28:31.160637+0000","last_fullsized":"2024-04-25T16:28:31.160637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:28:31.160547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:28:31.160547+0000","last_clean_scrub_stamp":"2024-04-25T16:28:31.160547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:35.566470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15613,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":31226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":884736,"data_stored":708870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":118010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":118046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":311296,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":286720,"data_stored":117920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":118080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":892928,"data_stored":708926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:28:50.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T16:28:50.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.818+0000 7faea5a0c640 1 Processor -- start 2024-04-25T16:28:50.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.818+0000 7faea5a0c640 1 -- start start 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7faea5a0c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faea00b3600 0x7faea00b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7faea5a0c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faea00d0340 0x7faea00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7faea5a0c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faea00d3740 con 0x7faea00cda40 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7faea5a0c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faea00cd5b0 con 0x7faea00b3600 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7faea5a0c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faea00cf140 con 0x7faea00d0340 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7fae9effd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faea00b3600 0x7faea00b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7fae9effd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faea00b3600 0x7faea00b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44290/0 (socket says 172.21.15.94:44290) 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.819+0000 7fae9effd640 1 -- 172.21.15.94:0/3503953016 learned_addr learned my addr 172.21.15.94:0/3503953016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae97fff640 1 --2- 172.21.15.94:0/3503953016 >> v2:172.21.15.138:3300/0 conn(0x7faea00d0340 0x7faea00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae9effd640 1 -- 172.21.15.94:0/3503953016 >> v2:172.21.15.138:3300/0 conn(0x7faea00d0340 msgr2=0x7faea00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae9effd640 1 --2- 172.21.15.94:0/3503953016 >> v2:172.21.15.138:3300/0 conn(0x7faea00d0340 0x7faea00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae9effd640 1 -- 172.21.15.94:0/3503953016 >> v1:172.21.15.138:6789/0 conn(0x7faea00cda40 legacy=0x7faea00cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae9effd640 1 -- 172.21.15.94:0/3503953016 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea00cf420 con 0x7faea00b3600 2024-04-25T16:28:50.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae9effd640 1 --2- 172.21.15.94:0/3503953016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00b3600 0x7faea00b4220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faea00d12f0 tx=0x7fae9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899cf8049a4c5780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:50.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.820+0000 7fae9e7fc640 1 -- 172.21.15.94:0/3503953016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90004030 con 0x7faea00b3600 2024-04-25T16:28:50.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7fae9e7fc640 1 -- 172.21.15.94:0/3503953016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae900041d0 con 0x7faea00b3600 2024-04-25T16:28:50.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7fae9e7fc640 1 -- 172.21.15.94:0/3503953016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90003610 con 0x7faea00b3600 2024-04-25T16:28:50.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 -- 172.21.15.94:0/3503953016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00b3600 msgr2=0x7faea00b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 --2- 172.21.15.94:0/3503953016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00b3600 0x7faea00b4220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faea00d12f0 tx=0x7fae9002a270 comp rx=0 tx=0).stop 2024-04-25T16:28:50.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 -- 172.21.15.94:0/3503953016 shutdown_connections 2024-04-25T16:28:50.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 --2- 172.21.15.94:0/3503953016 >> v2:172.21.15.138:3300/0 conn(0x7faea00d0340 0x7faea00b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 --2- 172.21.15.94:0/3503953016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00b3600 0x7faea00b4220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 -- 172.21.15.94:0/3503953016 >> 172.21.15.94:0/3503953016 conn(0x7faea00b7b20 msgr2=0x7faea00b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 -- 172.21.15.94:0/3503953016 shutdown_connections 2024-04-25T16:28:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.821+0000 7faea5a0c640 1 -- 172.21.15.94:0/3503953016 wait complete. 2024-04-25T16:28:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.823+0000 7faea5a0c640 1 Processor -- start 2024-04-25T16:28:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.823+0000 7faea5a0c640 1 -- start start 2024-04-25T16:28:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7faea5a0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea00b3600 0x7faea00b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7faea5a0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 0x7faea0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7faea5a0c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea00d0340 0x7faea01f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7faea5a0c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea00cfe30 con 0x7faea00cda40 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7faea5a0c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea00cfcb0 con 0x7faea00b3600 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7faea5a0c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faea00d3110 con 0x7faea00d0340 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7fae97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 0x7faea0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7fae97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 0x7faea0154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44292/0 (socket says 172.21.15.94:44292) 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.824+0000 7fae97fff640 1 -- 172.21.15.94:0/690847996 learned_addr learned my addr 172.21.15.94:0/690847996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae9effd640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea00b3600 0x7faea00b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae9f7fe640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea00d0340 0x7faea01f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae97fff640 1 -- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea00d0340 msgr2=0x7faea01f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae97fff640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea00d0340 0x7faea01f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae97fff640 1 -- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea00b3600 msgr2=0x7faea00b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:50.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae97fff640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea00b3600 0x7faea00b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae97fff640 1 -- 172.21.15.94:0/690847996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea00aff00 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae97fff640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 0x7faea0154be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fae8800d5d0 tx=0x7fae8800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae880125f0 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae88019070 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea00b01e0 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae880129f0 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.825+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea00b07a0 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.826+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea00afc70 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.827+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae88012b90 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.827+0000 7fae9cff9640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fae800744b0 0x7fae800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.827+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fae880959d0 con 0x7faea00cda40 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.827+0000 7fae9effd640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fae800744b0 0x7fae800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.828+0000 7fae9effd640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fae800744b0 0x7fae800769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae90033000 tx=0x7fae9002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:50.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:50.832+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae8805fdf0 con 0x7faea00cda40 2024-04-25T16:28:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.007+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7faea006a180 con 0x7faea00cda40 2024-04-25T16:28:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.010+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fae8805f6f0 con 0x7faea00cda40 2024-04-25T16:28:51.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.012+0000 7fae9cff9640 1 -- 172.21.15.94:0/690847996 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fae8805f200 con 0x7faea00cda40 2024-04-25T16:28:51.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fae800744b0 msgr2=0x7fae800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fae800744b0 0x7fae800769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae90033000 tx=0x7fae9002e000 comp rx=0 tx=0).stop 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 msgr2=0x7faea0154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 0x7faea0154be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fae8800d5d0 tx=0x7fae8800da90 comp rx=0 tx=0).stop 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 shutdown_connections 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fae800744b0 0x7fae800769c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea00d0340 0x7faea01f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea00cda40 0x7faea0154be0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 --2- 172.21.15.94:0/690847996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea00b3600 0x7faea00b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 >> 172.21.15.94:0/690847996 conn(0x7faea00b7b20 msgr2=0x7faea0071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 shutdown_connections 2024-04-25T16:28:51.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.014+0000 7faea5a0c640 1 -- 172.21.15.94:0/690847996 wait complete. 2024-04-25T16:28:51.026 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T16:28:51.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.132+0000 7f074776c640 1 Processor -- start 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.132+0000 7f074776c640 1 -- start start 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f074776c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 0x7f074006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f074776c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07400b44d0 con 0x7f07400d11b0 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f074776c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07400b47b0 con 0x7f07400b3570 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f074776c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07400b50e0 con 0x7f07400b3d60 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0744d07640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f07400d11b0 0x7f07400d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:34274/0 (socket says 172.21.15.94:34274) 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0744d07640 1 -- 172.21.15.94:0/4002037963 learned_addr learned my addr 172.21.15.94:0/4002037963 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0745d09640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 0x7f074006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2338134776 0 0) 0x7f07400b44d0 con 0x7f07400d11b0 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f072c003520 con 0x7f07400d11b0 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 575326418 0 0) 0x7f072c003520 con 0x7f07400d11b0 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 msgr2=0x7f074006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 0x7f074006ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 >> v1:172.21.15.138:6789/0 conn(0x7f07400b3570 legacy=0x7f07400d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07400b4a90 con 0x7f07400d11b0 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f0730001670 con 0x7f07400d11b0 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.134+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 >> v1:172.21.15.94:6789/0 conn(0x7f07400d11b0 legacy=0x7f07400d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0737fff640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f072c0039e0 0x7f072c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0737fff640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f072c004800 0x7f072c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0737fff640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 0x7f072c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f072c00ba70 con 0x7f072c0039e0 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f072c00bbf0 con 0x7f072c004800 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f072c00bdf0 con 0x7f07400b3d60 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745508640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f072c0039e0 0x7f072c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0744d07640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f072c004800 0x7f072c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745d09640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 0x7f072c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745508640 1 -- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 msgr2=0x7f072c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745508640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07400b3d60 0x7f072c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745508640 1 -- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f072c004800 msgr2=0x7f072c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745508640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f072c004800 0x7f072c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.135+0000 7f0745508640 1 -- 172.21.15.94:0/4002037963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c003520 con 0x7f072c0039e0 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f0745508640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f072c0039e0 0x7f072c003e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f073c001fb0 tx=0x7f072c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14443724c3f03623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c001b80 con 0x7f072c0039e0 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f073c031620 con 0x7f072c0039e0 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f0737fff640 1 -- 172.21.15.94:0/4002037963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c031a30 con 0x7f072c0039e0 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f074776c640 1 -- 172.21.15.94:0/4002037963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f072c0039e0 msgr2=0x7f072c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f074776c640 1 --2- 172.21.15.94:0/4002037963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f072c0039e0 0x7f072c003e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f073c001fb0 tx=0x7f072c005830 comp rx=0 tx=0).stop 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f0745d09640 1 -- 172.21.15.94:0/4002037963 reap_dead start 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f074776c640 1 -- 172.21.15.94:0/4002037963 shutdown_connections 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.136+0000 7f074776c640 1 -- 172.21.15.94:0/4002037963 >> 172.21.15.94:0/4002037963 conn(0x7f07400d4090 msgr2=0x7f07400d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.137+0000 7f074776c640 1 -- 172.21.15.94:0/4002037963 shutdown_connections 2024-04-25T16:28:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.137+0000 7f074776c640 1 -- 172.21.15.94:0/4002037963 wait complete. 2024-04-25T16:28:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.138+0000 7f074776c640 1 Processor -- start 2024-04-25T16:28:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.138+0000 7f074776c640 1 -- start start 2024-04-25T16:28:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f072c0039e0 0x7f07401f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 0x7f074014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074014c320 0x7f074015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f074006ef10 con 0x7f07400b6bc0 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07400b4170 con 0x7f074014c320 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f074006f090 con 0x7f072c0039e0 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 0x7f074014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 0x7f074014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44316/0 (socket says 172.21.15.94:44316) 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 -- 172.21.15.94:0/3107297562 learned_addr learned my addr 172.21.15.94:0/3107297562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0745d09640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074014c320 0x7f074015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0745508640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f072c0039e0 0x7f07401f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 -- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f072c0039e0 msgr2=0x7f07401f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f072c0039e0 0x7f07401f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 -- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074014c320 msgr2=0x7f074015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074014c320 0x7f074015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 -- 172.21.15.94:0/3107297562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07401f5f90 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0745508640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f072c0039e0 0x7f07401f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0744d07640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 0x7f074014bde0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f07300072f0 tx=0x7f07300077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073001a070 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f073000f960 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07400b7680 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.140+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073000fd50 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.141+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07400b7c40 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.142+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074006a180 con 0x7f07400b6bc0 2024-04-25T16:28:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.142+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f073001d070 con 0x7f07400b6bc0 2024-04-25T16:28:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.143+0000 7f0735ffb640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07240744b0 0x7f07240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.143+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0730094fe0 con 0x7f07400b6bc0 2024-04-25T16:28:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.143+0000 7f0745508640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07240744b0 0x7f07240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.143+0000 7f0745508640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07240744b0 0x7f07240769c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f073c034000 tx=0x7f073c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.148+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f073005f400 con 0x7f07400b6bc0 2024-04-25T16:28:51.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.363+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07400b7410 con 0x7f07400b6bc0 2024-04-25T16:28:51.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.363+0000 7f0735ffb640 1 -- 172.21.15.94:0/3107297562 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f07300673f0 con 0x7f07400b6bc0 2024-04-25T16:28:51.364 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:28:51.364 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:18.975101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:28:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07240744b0 msgr2=0x7f07240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07240744b0 0x7f07240769c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f073c034000 tx=0x7f073c02f000 comp rx=0 tx=0).stop 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 msgr2=0x7f074014bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 0x7f074014bde0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f07300072f0 tx=0x7f07300077b0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 shutdown_connections 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f07240744b0 0x7f07240769c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074014c320 0x7f074015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07400b6bc0 0x7f074014bde0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 --2- 172.21.15.94:0/3107297562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f072c0039e0 0x7f07401f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 >> 172.21.15.94:0/3107297562 conn(0x7f07400d4090 msgr2=0x7f07400b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 shutdown_connections 2024-04-25T16:28:51.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.365+0000 7f074776c640 1 -- 172.21.15.94:0/3107297562 wait complete. 2024-04-25T16:28:51.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:28:51.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.484+0000 7f52961e3640 1 Processor -- start 2024-04-25T16:28:51.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.484+0000 7f52961e3640 1 -- start start 2024-04-25T16:28:51.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.485+0000 7f52961e3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f52900d4bb0 0x7f529006da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.485+0000 7f52961e3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529006c9c0 con 0x7f529008e780 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.485+0000 7f52961e3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529006cca0 con 0x7f52900d1010 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.485+0000 7f52961e3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f529006dfa0 con 0x7f52900d4bb0 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.485+0000 7f528f7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f529008e780 0x7f52900b5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:41700/0 (socket says 172.21.15.94:41700) 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.485+0000 7f528f7fe640 1 -- 172.21.15.94:0/3557604042 learned_addr learned my addr 172.21.15.94:0/3557604042 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528effd640 1 --2- 172.21.15.94:0/3557604042 >> v2:172.21.15.138:3300/0 conn(0x7f52900d4bb0 0x7f529006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600009575 0 0) 0x7f529006c9c0 con 0x7f529008e780 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5278003520 con 0x7f529008e780 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993110346 0 0) 0x7f5278003520 con 0x7f529008e780 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 >> v2:172.21.15.138:3300/0 conn(0x7f52900d4bb0 msgr2=0x7f529006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 --2- 172.21.15.94:0/3557604042 >> v2:172.21.15.138:3300/0 conn(0x7f52900d4bb0 0x7f529006da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 >> v1:172.21.15.138:6789/0 conn(0x7f52900d1010 legacy=0x7f52900d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529006cf80 con 0x7f529008e780 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f5280001670 con 0x7f529008e780 2024-04-25T16:28:51.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 >> v1:172.21.15.94:6789/0 conn(0x7f529008e780 legacy=0x7f52900b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52780039b0 0x7f5278003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5278004800 0x7f5278004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5278005250 0x7f527800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f529006dfa0 con 0x7f5278005250 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f527800c1e0 con 0x7f52780039b0 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f527800c3e0 con 0x7f5278004800 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528effd640 1 --2- 172.21.15.94:0/3557604042 >> v2:172.21.15.138:3300/0 conn(0x7f52900d4bb0 0x7f529006da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528ffff640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5278005250 0x7f527800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528f7fe640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52780039b0 0x7f5278003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528effd640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5278004800 0x7f5278004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528ffff640 1 -- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5278004800 msgr2=0x7f5278004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528ffff640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5278004800 0x7f5278004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528ffff640 1 -- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52780039b0 msgr2=0x7f5278003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528ffff640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52780039b0 0x7f5278003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.486+0000 7f528ffff640 1 -- 172.21.15.94:0/3557604042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5278003520 con 0x7f5278005250 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.487+0000 7f528ffff640 1 -- 172.21.15.94:0/3557604042 reap_dead start 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.487+0000 7f528ffff640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5278005250 0x7f527800b9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5288001e50 tx=0x7f5278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328466505aa4d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.487+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288001a70 con 0x7f5278005250 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.487+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5288003970 con 0x7f5278005250 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.487+0000 7f528e7fc640 1 -- 172.21.15.94:0/3557604042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288003d80 con 0x7f5278005250 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 -- 172.21.15.94:0/3557604042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5278005250 msgr2=0x7f527800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5278005250 0x7f527800b9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5288001e50 tx=0x7f5278005830 comp rx=0 tx=0).stop 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 -- 172.21.15.94:0/3557604042 shutdown_connections 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 --2- 172.21.15.94:0/3557604042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5278005250 0x7f527800b9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 -- 172.21.15.94:0/3557604042 >> 172.21.15.94:0/3557604042 conn(0x7f52900b7cc0 msgr2=0x7f52900b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 -- 172.21.15.94:0/3557604042 shutdown_connections 2024-04-25T16:28:51.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.488+0000 7f52961e3640 1 -- 172.21.15.94:0/3557604042 wait complete. 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.490+0000 7f52961e3640 1 Processor -- start 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.490+0000 7f52961e3640 1 -- start start 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f52961e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5278005250 0x7f52900d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f52961e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529014ed50 0x7f529015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f52961e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 0x7f52901fa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f52961e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f529006c570 con 0x7f52901fa2a0 2024-04-25T16:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f52961e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f529006d540 con 0x7f5278005250 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f52961e3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f529006c6f0 con 0x7f529014ed50 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f528ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 0x7f52901fa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f528ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 0x7f52901fa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50486/0 (socket says 172.21.15.94:50486) 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.491+0000 7f528ffff640 1 -- 172.21.15.94:0/184906885 learned_addr learned my addr 172.21.15.94:0/184906885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528ffff640 1 -- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529014ed50 msgr2=0x7f529015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528ffff640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529014ed50 0x7f529015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528ffff640 1 -- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5278005250 msgr2=0x7f52900d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528f7fe640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5278005250 0x7f52900d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528ffff640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5278005250 0x7f52900d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528ffff640 1 -- 172.21.15.94:0/184906885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52900cd790 con 0x7f52901fa2a0 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f528ffff640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 0x7f52901fa6b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5288001f70 tx=0x7f52880014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288001a70 con 0x7f52901fa2a0 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f528802ac70 con 0x7f52901fa2a0 2024-04-25T16:28:51.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52901617c0 con 0x7f52901fa2a0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288006d90 con 0x7f52901fa2a0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.492+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5290161a60 con 0x7f52901fa2a0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.493+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529006a180 con 0x7f52901fa2a0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.494+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5288007030 con 0x7f52901fa2a0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.494+0000 7f5277fff640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f527c0744b0 0x7f527c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.494+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f52880b4f30 con 0x7f52901fa2a0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.494+0000 7f528f7fe640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f527c0744b0 0x7f527c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.495+0000 7f528f7fe640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f527c0744b0 0x7f527c0769c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f52800017b0 tx=0x7f5280009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.499+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f528807f2c0 con 0x7f52901fa2a0 2024-04-25T16:28:51.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.710+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52900d1420 con 0x7f52901fa2a0 2024-04-25T16:28:51.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.710+0000 7f5277fff640 1 -- 172.21.15.94:0/184906885 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f5288087340 con 0x7f52901fa2a0 2024-04-25T16:28:51.711 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:28:51.711 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/931278237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":931278237},{"type":"v1","addr":"172.21.15.138:6835","nonce":931278237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2939595128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2939595128},{"type":"v1","addr":"172.21.15.94:6836","nonce":2939595128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2318516936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2318516936},{"type":"v1","addr":"172.21.15.94:6837","nonce":2318516936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:26:55.900210+0000","modified":"2024-04-25T16:27:18.975101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6834/4275391000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4275391000},{"type":"v1","addr":"172.21.15.138:6834","nonce":4275391000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T16:28:51.711 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f527c0744b0 msgr2=0x7f527c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f527c0744b0 0x7f527c0769c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f52800017b0 tx=0x7f5280009040 comp rx=0 tx=0).stop 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 msgr2=0x7f52901fa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 0x7f52901fa6b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5288001f70 tx=0x7f52880014a0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 shutdown_connections 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f527c0744b0 0x7f527c0769c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52901fa2a0 0x7f52901fa6b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529014ed50 0x7f529015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 --2- 172.21.15.94:0/184906885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5278005250 0x7f52900d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.712+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 >> 172.21.15.94:0/184906885 conn(0x7f52900b7cc0 msgr2=0x7f52900ba820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.713+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 shutdown_connections 2024-04-25T16:28:51.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.713+0000 7f52961e3640 1 -- 172.21.15.94:0/184906885 wait complete. 2024-04-25T16:28:51.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:28:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.830+0000 7fa834f4d640 1 Processor -- start 2024-04-25T16:28:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.830+0000 7fa834f4d640 1 -- start start 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa834f4d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa8300b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa834f4d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa83006dc90 0x7fa8300b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa834f4d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8300b1650 con 0x7fa830161ab0 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa834f4d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa830161620 con 0x7fa83006dc90 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa834f4d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa830160fd0 con 0x7fa83015d3e0 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa82ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa83006dc90 0x7fa8300b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa82dd9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa830161ab0 0x7fa83006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:41714/0 (socket says 172.21.15.94:41714) 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.831+0000 7fa82dd9b640 1 -- 172.21.15.94:0/1277537724 learned_addr learned my addr 172.21.15.94:0/1277537724 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82e59c640 1 --2- 172.21.15.94:0/1277537724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa8300b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82ed9d640 1 -- 172.21.15.94:0/1277537724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 msgr2=0x7fa8300b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82ed9d640 1 --2- 172.21.15.94:0/1277537724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa8300b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82ed9d640 1 -- 172.21.15.94:0/1277537724 >> v1:172.21.15.94:6789/0 conn(0x7fa830161ab0 legacy=0x7fa83006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82ed9d640 1 -- 172.21.15.94:0/1277537724 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8301612b0 con 0x7fa83006dc90 2024-04-25T16:28:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82d59a640 1 -- 172.21.15.94:0/1277537724 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072930050 0 0) 0x7fa8300b1650 con 0x7fa830161ab0 2024-04-25T16:28:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.832+0000 7fa82ed9d640 1 --2- 172.21.15.94:0/1277537724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83006dc90 0x7fa8300b5550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa828009d90 tx=0x7fa82802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29eafb635ceb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa82d59a640 1 -- 172.21.15.94:0/1277537724 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828007740 con 0x7fa83006dc90 2024-04-25T16:28:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa82d59a640 1 -- 172.21.15.94:0/1277537724 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa8280078e0 con 0x7fa83006dc90 2024-04-25T16:28:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa82d59a640 1 -- 172.21.15.94:0/1277537724 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828007cd0 con 0x7fa83006dc90 2024-04-25T16:28:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 -- 172.21.15.94:0/1277537724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83006dc90 msgr2=0x7fa8300b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 --2- 172.21.15.94:0/1277537724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83006dc90 0x7fa8300b5550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa828009d90 tx=0x7fa82802a680 comp rx=0 tx=0).stop 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 -- 172.21.15.94:0/1277537724 shutdown_connections 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 --2- 172.21.15.94:0/1277537724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83006dc90 0x7fa8300b5550 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 --2- 172.21.15.94:0/1277537724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa8300b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 -- 172.21.15.94:0/1277537724 >> 172.21.15.94:0/1277537724 conn(0x7fa8300d23b0 msgr2=0x7fa830073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 -- 172.21.15.94:0/1277537724 shutdown_connections 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.833+0000 7fa834f4d640 1 -- 172.21.15.94:0/1277537724 wait complete. 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.835+0000 7fa834f4d640 1 Processor -- start 2024-04-25T16:28:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.835+0000 7fa834f4d640 1 -- start start 2024-04-25T16:28:51.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa83006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa830161ab0 0x7fa83006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 0x7fa830142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8300b6af0 con 0x7fa83015a2c0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8300b6970 con 0x7fa830161ab0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8300b6c70 con 0x7fa83015d3e0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 0x7fa830142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 0x7fa830142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50492/0 (socket says 172.21.15.94:50492) 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 -- 172.21.15.94:0/2631203483 learned_addr learned my addr 172.21.15.94:0/2631203483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82e59c640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa83006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82dd9b640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa830161ab0 0x7fa83006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 -- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 msgr2=0x7fa83006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa83006ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 -- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa830161ab0 msgr2=0x7fa83006d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa830161ab0 0x7fa83006d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 -- 172.21.15.94:0/2631203483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83015a9d0 con 0x7fa83015a2c0 2024-04-25T16:28:51.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82e59c640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa83006ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82dd9b640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa830161ab0 0x7fa83006d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa82ed9d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 0x7fa830142210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa828009d90 tx=0x7fa828004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82803f070 con 0x7fa83015a2c0 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa828031c40 con 0x7fa83015a2c0 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83015acb0 con 0x7fa83015a2c0 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828003990 con 0x7fa83015a2c0 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.837+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83015b270 con 0x7fa83015a2c0 2024-04-25T16:28:51.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.838+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa830071340 con 0x7fa83015a2c0 2024-04-25T16:28:51.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.839+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa828003b30 con 0x7fa83015a2c0 2024-04-25T16:28:51.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.839+0000 7fa81effd640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8100744b0 0x7fa8100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.840+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa8280b6700 con 0x7fa83015a2c0 2024-04-25T16:28:51.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.840+0000 7fa82e59c640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8100744b0 0x7fa8100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.840+0000 7fa82e59c640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8100744b0 0x7fa8100769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:51.844+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa830071340 con 0x7fa83015a2c0 2024-04-25T16:28:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.024+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa83006a180 con 0x7fa83015a2c0 2024-04-25T16:28:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.025+0000 7fa81effd640 1 -- 172.21.15.94:0/2631203483 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18868 (secure 0 0 0) 0x7fa828080a10 con 0x7fa83015a2c0 2024-04-25T16:28:52.025 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:28:52.025 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"fsid":"e3c627c0-5913-4b5c-b1aa-d445103b794b","created":"2024-04-25T16:26:31.199303+0000","modified":"2024-04-25T16:27:36.681885+0000","last_up_change":"2024-04-25T16:26:42.487307+0000","last_in_change":"2024-04-25T16:26:33.468939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:26:43.312109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:26:53.603582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:26:54.860927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:26:58.564353+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dfe00173-43d3-4c80-aafc-55f29fe7e2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6809","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6811","nonce":2433651201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6813","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6815","nonce":2433651201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6821","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6823","nonce":2433651201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6817","nonce":2433651201},{"type":"v1","addr":"172.21.15.94:6819","nonce":2433651201}]},"public_addr":"172.21.15.94:6811/2433651201","cluster_addr":"172.21.15.94:6815/2433651201","heartbeat_back_addr":"172.21.15.94:6823/2433651201","heartbeat_front_addr":"172.21.15.94:6819/2433651201","state":["exists","up"]},{"osd":1,"uuid":"e1d86461-a4cb-42a0-a3c1-9772fdbdf564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6801","nonce":816278269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6803","nonce":816278269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6807","nonce":816278269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":816278269},{"type":"v1","addr":"172.21.15.94:6805","nonce":816278269}]},"public_addr":"172.21.15.94:6801/816278269","cluster_addr":"172.21.15.94:6803/816278269","heartbeat_back_addr":"172.21.15.94:6807/816278269","heartbeat_front_addr":"172.21.15.94:6805/816278269","state":["exists","up"]},{"osd":2,"uuid":"a76ea96d-c603-4215-b852-86446453a829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6825","nonce":2599289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6827","nonce":2599289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6831","nonce":2599289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2599289524},{"type":"v1","addr":"172.21.15.94:6829","nonce":2599289524}]},"public_addr":"172.21.15.94:6825/2599289524","cluster_addr":"172.21.15.94:6827/2599289524","heartbeat_back_addr":"172.21.15.94:6831/2599289524","heartbeat_front_addr":"172.21.15.94:6829/2599289524","state":["exists","up"]},{"osd":3,"uuid":"4b9e445a-59c0-4623-b982-7c8f29b19a53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6810","nonce":1336839764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6814","nonce":1336839764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6822","nonce":1336839764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1336839764},{"type":"v1","addr":"172.21.15.94:6818","nonce":1336839764}]},"public_addr":"172.21.15.94:6810/1336839764","cluster_addr":"172.21.15.94:6814/1336839764","heartbeat_back_addr":"172.21.15.94:6822/1336839764","heartbeat_front_addr":"172.21.15.94:6818/1336839764","state":["exists","up"]},{"osd":4,"uuid":"2a101628-de9f-4b35-93a7-97d72c9bf949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6817","nonce":3435025595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6819","nonce":3435025595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6823","nonce":3435025595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3435025595},{"type":"v1","addr":"172.21.15.138:6821","nonce":3435025595}]},"public_addr":"172.21.15.138:6817/3435025595","cluster_addr":"172.21.15.138:6819/3435025595","heartbeat_back_addr":"172.21.15.138:6823/3435025595","heartbeat_front_addr":"172.21.15.138:6821/3435025595","state":["exists","up"]},{"osd":5,"uuid":"9e2df0c1-56a4-4213-b625-c803b52fd641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6825","nonce":4211896878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6827","nonce":4211896878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6831","nonce":4211896878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4211896878},{"type":"v1","addr":"172.21.15.138:6829","nonce":4211896878}]},"public_addr":"172.21.15.138:6825/4211896878","cluster_addr":"172.21.15.138:6827/4211896878","heartbeat_back_addr":"172.21.15.138:6831/4211896878","heartbeat_front_addr":"172.21.15.138:6829/4211896878","state":["exists","up"]},{"osd":6,"uuid":"5fcc96fa-a293-459a-bc53-feeac42e222c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6801","nonce":62635939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6803","nonce":62635939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6807","nonce":62635939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":62635939},{"type":"v1","addr":"172.21.15.138:6805","nonce":62635939}]},"public_addr":"172.21.15.138:6801/62635939","cluster_addr":"172.21.15.138:6803/62635939","heartbeat_back_addr":"172.21.15.138:6807/62635939","heartbeat_front_addr":"172.21.15.138:6805/62635939","state":["exists","up"]},{"osd":7,"uuid":"6377ea75-413c-4635-baf1-69b26cc937b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6809","nonce":300369448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6811","nonce":300369448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6815","nonce":300369448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":300369448},{"type":"v1","addr":"172.21.15.138:6813","nonce":300369448}]},"public_addr":"172.21.15.138:6809/300369448","cluster_addr":"172.21.15.138:6811/300369448","heartbeat_back_addr":"172.21.15.138:6815/300369448","heartbeat_front_addr":"172.21.15.138:6813/300369448","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.622004+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.582517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.689210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.612892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.635787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.647729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.566440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:26:38.592239+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:28:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8100744b0 msgr2=0x7fa8100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8100744b0 0x7fa8100769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820008040 comp rx=0 tx=0).stop 2024-04-25T16:28:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 msgr2=0x7fa830142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 0x7fa830142210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa828009d90 tx=0x7fa828004a40 comp rx=0 tx=0).stop 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 shutdown_connections 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7fa8100744b0 0x7fa8100769c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa83015a2c0 0x7fa830142210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa830161ab0 0x7fa83006d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 --2- 172.21.15.94:0/2631203483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa83015d3e0 0x7fa83006ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 >> 172.21.15.94:0/2631203483 conn(0x7fa8300d23b0 msgr2=0x7fa8300b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 shutdown_connections 2024-04-25T16:28:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.027+0000 7fa834f4d640 1 -- 172.21.15.94:0/2631203483 wait complete. 2024-04-25T16:28:52.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T16:28:52.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.147+0000 7f562684c640 1 Processor -- start 2024-04-25T16:28:52.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.147+0000 7f562684c640 1 -- start start 2024-04-25T16:28:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.148+0000 7f562684c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 0x7f562006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.148+0000 7f562684c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56200d3b80 con 0x7f5620165350 2024-04-25T16:28:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.148+0000 7f562684c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f562009d6d0 con 0x7f562006e630 2024-04-25T16:28:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.148+0000 7f562684c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f562009d3f0 con 0x7f56200b97e0 2024-04-25T16:28:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.148+0000 7f5624de9640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f5620165350 0x7f56200ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:41724/0 (socket says 172.21.15.94:41724) 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.148+0000 7f5624de9640 1 -- 172.21.15.94:0/1718069587 learned_addr learned my addr 172.21.15.94:0/1718069587 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291196042 0 0) 0x7f56200d3b80 con 0x7f5620165350 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5608003520 con 0x7f5620165350 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561ffff640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 0x7f562006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3661569735 0 0) 0x7f5608003520 con 0x7f5620165350 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 msgr2=0x7f562006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 0x7f562006e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 >> v1:172.21.15.138:6789/0 conn(0x7f562006e630 legacy=0x7f5620164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56200d3530 con 0x7f5620165350 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1879525064 0 0) 0x7f56180019f0 con 0x7f5620165350 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 >> v1:172.21.15.94:6789/0 conn(0x7f5620165350 legacy=0x7f56200ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080039e0 0x7f5608003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5608004800 0x7f5608004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 0x7f5608005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f562009d3f0 con 0x7f5608004800 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f560800baf0 con 0x7f56080039e0 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f560800bcf0 con 0x7f56200b97e0 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561f7fe640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5608004800 0x7f5608004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561ffff640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080039e0 0x7f5608003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f5624de9640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 0x7f5608005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561f7fe640 1 -- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 msgr2=0x7f5608005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561f7fe640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56200b97e0 0x7f5608005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561f7fe640 1 -- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080039e0 msgr2=0x7f5608003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561f7fe640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080039e0 0x7f5608003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.149+0000 7f561f7fe640 1 -- 172.21.15.94:0/1718069587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5608003520 con 0x7f5608004800 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.150+0000 7f561f7fe640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5608004800 0x7f5608004c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f56200d2170 tx=0x7f5608005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804749d70f66ac46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.150+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614001370 con 0x7f5608004800 2024-04-25T16:28:52.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.150+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f56140052d0 con 0x7f5608004800 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.150+0000 7f561effd640 1 -- 172.21.15.94:0/1718069587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561402abb0 con 0x7f5608004800 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f562684c640 1 -- 172.21.15.94:0/1718069587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5608004800 msgr2=0x7f5608004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f562684c640 1 --2- 172.21.15.94:0/1718069587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5608004800 0x7f5608004c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f56200d2170 tx=0x7f5608005830 comp rx=0 tx=0).stop 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f5624de9640 1 -- 172.21.15.94:0/1718069587 reap_dead start 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f562684c640 1 -- 172.21.15.94:0/1718069587 shutdown_connections 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f562684c640 1 -- 172.21.15.94:0/1718069587 >> 172.21.15.94:0/1718069587 conn(0x7f56200b5a70 msgr2=0x7f56200b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f562684c640 1 -- 172.21.15.94:0/1718069587 shutdown_connections 2024-04-25T16:28:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.151+0000 7f562684c640 1 -- 172.21.15.94:0/1718069587 wait complete. 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.153+0000 7f562684c640 1 Processor -- start 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.153+0000 7f562684c640 1 -- start start 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f562684c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5608004800 0x7f56200cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f562684c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562015f660 0x7f56200c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f562684c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 0x7f56201612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f562684c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56200b2bf0 con 0x7f56200c7f60 2024-04-25T16:28:52.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f562684c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f562006f4e0 con 0x7f562015f660 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f562684c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56200b2d70 con 0x7f5608004800 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f5624de9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 0x7f56201612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f5624de9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 0x7f56201612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50516/0 (socket says 172.21.15.94:50516) 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f5624de9640 1 -- 172.21.15.94:0/606813280 learned_addr learned my addr 172.21.15.94:0/606813280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f561f7fe640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562015f660 0x7f56200c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f5624de9640 1 -- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5608004800 msgr2=0x7f56200cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.154+0000 7f561ffff640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5608004800 0x7f56200cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f5624de9640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5608004800 0x7f56200cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f5624de9640 1 -- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562015f660 msgr2=0x7f56200c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f5624de9640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562015f660 0x7f56200c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f5624de9640 1 -- 172.21.15.94:0/606813280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56200c8370 con 0x7f56200c7f60 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f561f7fe640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562015f660 0x7f56200c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f561ffff640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5608004800 0x7f56200cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:28:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f5624de9640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 0x7f56201612f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56200b6620 tx=0x7f5618007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561800ac80 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5618018070 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56200c9dc0 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618013650 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.155+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56200ca2c0 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.156+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56200ca030 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.156+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f56180137f0 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.157+0000 7f561cff9640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f560c0744b0 0x7f560c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.157+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5618095e20 con 0x7f56200c7f60 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.157+0000 7f561ffff640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f560c0744b0 0x7f560c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.157+0000 7f561ffff640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f560c0744b0 0x7f560c0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f562008d1e0 tx=0x7f5610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.161+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56200c8370 con 0x7f56200c7f60 2024-04-25T16:28:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.373+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f562006a180 con 0x7f56200c7f60 2024-04-25T16:28:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.375+0000 7f561cff9640 1 -- 172.21.15.94:0/606813280 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f56180601b0 con 0x7f56200c7f60 2024-04-25T16:28:52.376 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f560c0744b0 msgr2=0x7f560c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f560c0744b0 0x7f560c0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f562008d1e0 tx=0x7f5610008040 comp rx=0 tx=0).stop 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 msgr2=0x7f56201612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 0x7f56201612f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56200b6620 tx=0x7f5618007bd0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 shutdown_connections 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f560c0744b0 0x7f560c0769c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56200c7f60 0x7f56201612f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562015f660 0x7f56200c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.378+0000 7f562684c640 1 --2- 172.21.15.94:0/606813280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5608004800 0x7f56200cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.379+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 >> 172.21.15.94:0/606813280 conn(0x7f56200b5a70 msgr2=0x7f5620160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.379+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 shutdown_connections 2024-04-25T16:28:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.379+0000 7f562684c640 1 -- 172.21.15.94:0/606813280 wait complete. 2024-04-25T16:28:52.382 INFO:tasks.ceph.mds.d.smithi138.stderr: -8817> 2024-04-25T16:26:49.061+0000 7fa302be6640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6833/0: (98) Address already in use 2024-04-25T16:28:52.383 INFO:tasks.ceph.mds.d.smithi138.stderr: -7616> 2024-04-25T16:27:00.062+0000 7fa2fcbda640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T16:28:52.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T16:28:52.444 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-04-25T16:28:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.502+0000 7f4db39ab640 1 Processor -- start 2024-04-25T16:28:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.502+0000 7f4db39ab640 1 -- start start 2024-04-25T16:28:52.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db39ab640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4dac0b4c70 0x7f4dac160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db39ab640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4dac0d4a20 0x7f4dac0d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db39ab640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dac161930 con 0x7f4dac0b5710 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db39ab640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4dac161c10 con 0x7f4dac0b4c70 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db39ab640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4dac164d10 con 0x7f4dac0d4a20 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db1747640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4dac0b5710 0x7f4dac1608e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:41726/0 (socket says 172.21.15.94:41726) 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db1747640 1 -- 172.21.15.94:0/117055551 learned_addr learned my addr 172.21.15.94:0/117055551 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.503+0000 7f4db0f46640 1 --2- 172.21.15.94:0/117055551 >> v2:172.21.15.94:3300/0 conn(0x7f4dac0b4c70 0x7f4dac160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db1f48640 1 --2- 172.21.15.94:0/117055551 >> v2:172.21.15.138:3300/0 conn(0x7f4dac0d4a20 0x7f4dac0d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4da3fff640 1 -- 172.21.15.94:0/117055551 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093962459 0 0) 0x7f4dac161930 con 0x7f4dac0b5710 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4da3fff640 1 -- 172.21.15.94:0/117055551 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d98003520 con 0x7f4dac0b5710 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db0f46640 1 -- 172.21.15.94:0/117055551 >> v2:172.21.15.138:3300/0 conn(0x7f4dac0d4a20 msgr2=0x7f4dac0d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db0f46640 1 --2- 172.21.15.94:0/117055551 >> v2:172.21.15.138:3300/0 conn(0x7f4dac0d4a20 0x7f4dac0d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db0f46640 1 -- 172.21.15.94:0/117055551 >> v1:172.21.15.94:6789/0 conn(0x7f4dac0b5710 legacy=0x7f4dac1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db0f46640 1 -- 172.21.15.94:0/117055551 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac164ff0 con 0x7f4dac0b4c70 2024-04-25T16:28:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db1f48640 1 --2- 172.21.15.94:0/117055551 >> v2:172.21.15.138:3300/0 conn(0x7f4dac0d4a20 0x7f4dac0d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:28:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.504+0000 7f4db0f46640 1 --2- 172.21.15.94:0/117055551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b4c70 0x7f4dac160ff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0d1e80 tx=0x7f4d9c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfa5e73feb81cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4da3fff640 1 -- 172.21.15.94:0/117055551 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c037070 con 0x7f4dac0b4c70 2024-04-25T16:28:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4da3fff640 1 -- 172.21.15.94:0/117055551 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4d9c02add0 con 0x7f4dac0b4c70 2024-04-25T16:28:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4da3fff640 1 -- 172.21.15.94:0/117055551 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c032680 con 0x7f4dac0b4c70 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 -- 172.21.15.94:0/117055551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b4c70 msgr2=0x7f4dac160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 --2- 172.21.15.94:0/117055551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b4c70 0x7f4dac160ff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0d1e80 tx=0x7f4d9c02a080 comp rx=0 tx=0).stop 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 -- 172.21.15.94:0/117055551 shutdown_connections 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 --2- 172.21.15.94:0/117055551 >> v2:172.21.15.138:3300/0 conn(0x7f4dac0d4a20 0x7f4dac0d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 --2- 172.21.15.94:0/117055551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b4c70 0x7f4dac160ff0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 -- 172.21.15.94:0/117055551 >> 172.21.15.94:0/117055551 conn(0x7f4dac0d6930 msgr2=0x7f4dac0b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 -- 172.21.15.94:0/117055551 shutdown_connections 2024-04-25T16:28:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.505+0000 7f4db39ab640 1 -- 172.21.15.94:0/117055551 wait complete. 2024-04-25T16:28:52.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.507+0000 7f4db39ab640 1 Processor -- start 2024-04-25T16:28:52.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.507+0000 7f4db39ab640 1 -- start start 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.508+0000 7f4db39ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dac0b4c70 0x7f4dac1f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.508+0000 7f4db39ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 0x7f4dac0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.508+0000 7f4db39ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4dac0d4a20 0x7f4dac0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.508+0000 7f4db39ab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac161530 con 0x7f4dac0b5710 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.508+0000 7f4db39ab640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac0b51c0 con 0x7f4dac0b4c70 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.508+0000 7f4db39ab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4dac1616b0 con 0x7f4dac0d4a20 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 0x7f4dac0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 0x7f4dac0cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50542/0 (socket says 172.21.15.94:50542) 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 -- 172.21.15.94:0/2141603939 learned_addr learned my addr 172.21.15.94:0/2141603939 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db1747640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dac0b4c70 0x7f4dac1f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db1f48640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4dac0d4a20 0x7f4dac0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 -- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4dac0d4a20 msgr2=0x7f4dac0cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4dac0d4a20 0x7f4dac0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 -- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dac0b4c70 msgr2=0x7f4dac1f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dac0b4c70 0x7f4dac1f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 -- 172.21.15.94:0/2141603939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac1ea160 con 0x7f4dac0b5710 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db0f46640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 0x7f4dac0cba80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009890 tx=0x7f4d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c037040 con 0x7f4dac0b5710 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4d9c042070 con 0x7f4dac0b5710 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dac1ea440 con 0x7f4dac0b5710 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c03d680 con 0x7f4dac0b5710 2024-04-25T16:28:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.509+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dac1f6d30 con 0x7f4dac0b5710 2024-04-25T16:28:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.511+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dac0cc5b0 con 0x7f4dac0b5710 2024-04-25T16:28:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.511+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4d9c03d820 con 0x7f4dac0b5710 2024-04-25T16:28:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.511+0000 7f4da1ffb640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4d900744b0 0x7f4d900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.511+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f4d9c0b5d00 con 0x7f4dac0b5710 2024-04-25T16:28:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.511+0000 7f4db1747640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4d900744b0 0x7f4d900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.512+0000 7f4db1747640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4d900744b0 0x7f4d900769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4da8000f60 tx=0x7f4da8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.516+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d9c080030 con 0x7f4dac0b5710 2024-04-25T16:28:52.545 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:52.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:52.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:52.736+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4dac06a180 con 0x7f4dac0b5710 2024-04-25T16:28:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.377+0000 7f4da1ffb640 1 -- 172.21.15.94:0/2141603939 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f4d9c07f930 con 0x7f4dac0b5710 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4d900744b0 msgr2=0x7f4d900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4d900744b0 0x7f4d900769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4da8000f60 tx=0x7f4da8009040 comp rx=0 tx=0).stop 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 msgr2=0x7f4dac0cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 0x7f4dac0cba80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009890 tx=0x7f4d9c0046c0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 shutdown_connections 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f4d900744b0 0x7f4d900769c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4dac0d4a20 0x7f4dac0cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dac0b5710 0x7f4dac0cba80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.380+0000 7f4db39ab640 1 --2- 172.21.15.94:0/2141603939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dac0b4c70 0x7f4dac1f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.381+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 >> 172.21.15.94:0/2141603939 conn(0x7f4dac0d6930 msgr2=0x7f4dac1608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.381+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 shutdown_connections 2024-04-25T16:28:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.381+0000 7f4db39ab640 1 -- 172.21.15.94:0/2141603939 wait complete. 2024-04-25T16:28:53.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T16:28:53.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.507+0000 7f8d07469640 1 Processor -- start 2024-04-25T16:28:53.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.507+0000 7f8d07469640 1 -- start start 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.508+0000 7f8d07469640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d0009bfa0 0x7f8d000b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.508+0000 7f8d07469640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d000ced80 0x7f8d000cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.508+0000 7f8d07469640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d000d3040 con 0x7f8d000afc00 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.508+0000 7f8d07469640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8d000ce1e0 con 0x7f8d000ced80 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.508+0000 7f8d07469640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8d000b0010 con 0x7f8d0009bfa0 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d000ced80 0x7f8d000cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d000ced80 0x7f8d000cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50544/0 (socket says 172.21.15.94:50544) 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 -- 172.21.15.94:0/3343047015 learned_addr learned my addr 172.21.15.94:0/3343047015 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05205640 1 --2- 172.21.15.94:0/3343047015 >> v2:172.21.15.138:3300/0 conn(0x7f8d0009bfa0 0x7f8d000b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 -- 172.21.15.94:0/3343047015 >> v2:172.21.15.138:3300/0 conn(0x7f8d0009bfa0 msgr2=0x7f8d000b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 --2- 172.21.15.94:0/3343047015 >> v2:172.21.15.138:3300/0 conn(0x7f8d0009bfa0 0x7f8d000b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 -- 172.21.15.94:0/3343047015 >> v1:172.21.15.94:6789/0 conn(0x7f8d000afc00 legacy=0x7f8d000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8d05a06640 1 -- 172.21.15.94:0/3343047015 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d000d3320 con 0x7f8d000ced80 2024-04-25T16:28:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.509+0000 7f8cf7fff640 1 -- 172.21.15.94:0/3343047015 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114881339 0 0) 0x7f8d000d3040 con 0x7f8d000afc00 2024-04-25T16:28:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.510+0000 7f8d05a06640 1 --2- 172.21.15.94:0/3343047015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000cf590 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8d00074c20 tx=0x7f8cf802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29d52e802ebb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.510+0000 7f8cf7fff640 1 -- 172.21.15.94:0/3343047015 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8036070 con 0x7f8d000ced80 2024-04-25T16:28:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.510+0000 7f8cf7fff640 1 -- 172.21.15.94:0/3343047015 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8cf8031600 con 0x7f8d000ced80 2024-04-25T16:28:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.510+0000 7f8cf7fff640 1 -- 172.21.15.94:0/3343047015 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80319f0 con 0x7f8d000ced80 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.510+0000 7f8d07469640 1 -- 172.21.15.94:0/3343047015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 msgr2=0x7f8d000cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.510+0000 7f8d07469640 1 --2- 172.21.15.94:0/3343047015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000cf590 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8d00074c20 tx=0x7f8cf802a730 comp rx=0 tx=0).stop 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.511+0000 7f8d07469640 1 -- 172.21.15.94:0/3343047015 shutdown_connections 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.511+0000 7f8d07469640 1 --2- 172.21.15.94:0/3343047015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000cf590 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.511+0000 7f8d07469640 1 --2- 172.21.15.94:0/3343047015 >> v2:172.21.15.138:3300/0 conn(0x7f8d0009bfa0 0x7f8d000b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.511+0000 7f8d07469640 1 -- 172.21.15.94:0/3343047015 >> 172.21.15.94:0/3343047015 conn(0x7f8d00074810 msgr2=0x7f8d000b7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.511+0000 7f8d07469640 1 -- 172.21.15.94:0/3343047015 shutdown_connections 2024-04-25T16:28:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.511+0000 7f8d07469640 1 -- 172.21.15.94:0/3343047015 wait complete. 2024-04-25T16:28:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.512+0000 7f8d07469640 1 Processor -- start 2024-04-25T16:28:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.512+0000 7f8d07469640 1 -- start start 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0009bfa0 0x7f8d0015e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d000afc00 0x7f8d0015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000cf190 con 0x7f8d000ced80 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000cde70 con 0x7f8d0009bfa0 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d000cf310 con 0x7f8d000afc00 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50550/0 (socket says 172.21.15.94:50550) 2024-04-25T16:28:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 -- 172.21.15.94:0/3849437289 learned_addr learned my addr 172.21.15.94:0/3849437289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05205640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0009bfa0 0x7f8d0015e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d04a04640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d000afc00 0x7f8d0015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 -- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d000afc00 msgr2=0x7f8d0015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d000afc00 0x7f8d0015dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 -- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0009bfa0 msgr2=0x7f8d0015e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0009bfa0 0x7f8d0015e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 -- 172.21.15.94:0/3849437289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d000cb860 con 0x7f8d000ced80 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d05a06640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000b3f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8034040 tx=0x7f8cf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8036050 con 0x7f8d000ced80 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8cf8042070 con 0x7f8d000ced80 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d000cbb40 con 0x7f8d000ced80 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.514+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8009510 con 0x7f8d000ced80 2024-04-25T16:28:53.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.515+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d001f6bc0 con 0x7f8d000ced80 2024-04-25T16:28:53.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.516+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0006a180 con 0x7f8d000ced80 2024-04-25T16:28:53.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.516+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8cf8007c10 con 0x7f8d000ced80 2024-04-25T16:28:53.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.517+0000 7f8cf5ffb640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8ce40744b0 0x7f8ce40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.517+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f8cf80b6610 con 0x7f8d000ced80 2024-04-25T16:28:53.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.517+0000 7f8d05205640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8ce40744b0 0x7f8ce40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.517+0000 7f8d05205640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8ce40744b0 0x7f8ce40769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc006fd0 tx=0x7f8cfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:53.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.521+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cf8080940 con 0x7f8d000ced80 2024-04-25T16:28:53.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:53.693+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d000b7bf0 con 0x7f8d000ced80 2024-04-25T16:28:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.379+0000 7f8cf5ffb640 1 -- 172.21.15.94:0/3849437289 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f8cf8080240 con 0x7f8d000ced80 2024-04-25T16:28:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-04-25T16:28:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8ce40744b0 msgr2=0x7f8ce40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8ce40744b0 0x7f8ce40769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc006fd0 tx=0x7f8cfc008040 comp rx=0 tx=0).stop 2024-04-25T16:28:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 msgr2=0x7f8d000b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000b3f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8034040 tx=0x7f8cf8004a40 comp rx=0 tx=0).stop 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 shutdown_connections 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f8ce40744b0 0x7f8ce40769c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d000ced80 0x7f8d000b3f80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d000afc00 0x7f8d0015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 --2- 172.21.15.94:0/3849437289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0009bfa0 0x7f8d0015e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.385+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 >> 172.21.15.94:0/3849437289 conn(0x7f8d00074810 msgr2=0x7f8d00160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.386+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 shutdown_connections 2024-04-25T16:28:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.386+0000 7f8d07469640 1 -- 172.21.15.94:0/3849437289 wait complete. 2024-04-25T16:28:54.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T16:28:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.518+0000 7f3c0dd2e640 1 Processor -- start 2024-04-25T16:28:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.518+0000 7f3c0dd2e640 1 -- start start 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c0dd2e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c0809bfa0 0x7f3c080b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c0dd2e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c080afc00 0x7f3c080ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c0dd2e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c080d3040 con 0x7f3c080cebb0 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c0dd2e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3c080ce1e0 con 0x7f3c080afc00 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c0dd2e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3c080b0010 con 0x7f3c0809bfa0 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c06ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c080afc00 0x7f3c080ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c06ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c080afc00 0x7f3c080ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50564/0 (socket says 172.21.15.94:50564) 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.519+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1167177115 learned_addr learned my addr 172.21.15.94:0/1167177115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c077fe640 1 --2- 172.21.15.94:0/1167177115 >> v2:172.21.15.138:3300/0 conn(0x7f3c0809bfa0 0x7f3c080b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1167177115 >> v2:172.21.15.138:3300/0 conn(0x7f3c0809bfa0 msgr2=0x7f3c080b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:54.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/1167177115 >> v2:172.21.15.138:3300/0 conn(0x7f3c0809bfa0 0x7f3c080b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1167177115 >> v1:172.21.15.138:6789/0 conn(0x7f3c080cebb0 legacy=0x7f3c080d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:54.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1167177115 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c080d3320 con 0x7f3c080afc00 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/1167177115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c080afc00 0x7f3c080ce670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3c08074c20 tx=0x7f3bf802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66b0a4fa13b057f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.520+0000 7f3c067fc640 1 -- 172.21.15.94:0/1167177115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8037070 con 0x7f3c080afc00 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c067fc640 1 -- 172.21.15.94:0/1167177115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3bf8032440 con 0x7f3c080afc00 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c067fc640 1 -- 172.21.15.94:0/1167177115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8032830 con 0x7f3c080afc00 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/1167177115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c080afc00 msgr2=0x7f3c080ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/1167177115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c080afc00 0x7f3c080ce670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3c08074c20 tx=0x7f3bf802a230 comp rx=0 tx=0).stop 2024-04-25T16:28:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/1167177115 shutdown_connections 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/1167177115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c080afc00 0x7f3c080ce670 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/1167177115 >> v2:172.21.15.138:3300/0 conn(0x7f3c0809bfa0 0x7f3c080b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/1167177115 >> 172.21.15.94:0/1167177115 conn(0x7f3c08074810 msgr2=0x7f3c080b7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/1167177115 shutdown_connections 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.521+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/1167177115 wait complete. 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.523+0000 7f3c0dd2e640 1 Processor -- start 2024-04-25T16:28:54.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.523+0000 7f3c0dd2e640 1 -- start start 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c0dd2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 0x7f3c08152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c0dd2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c080afc00 0x7f3c080cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c0dd2e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c080cebb0 0x7f3c0806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c0dd2e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c080d05a0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c0dd2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c080cde70 con 0x7f3c080afc00 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c0dd2e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c080cefc0 con 0x7f3c080cebb0 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 0x7f3c08152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 0x7f3c08152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50568/0 (socket says 172.21.15.94:50568) 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.524+0000 7f3c077fe640 1 -- 172.21.15.94:0/606962702 learned_addr learned my addr 172.21.15.94:0/606962702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c080afc00 0x7f3c080cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c07fff640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c080cebb0 0x7f3c0806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c077fe640 1 -- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c080cebb0 msgr2=0x7f3c0806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c077fe640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c080cebb0 0x7f3c0806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c077fe640 1 -- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c080afc00 msgr2=0x7f3c080cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c077fe640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c080afc00 0x7f3c080cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c077fe640 1 -- 172.21.15.94:0/606962702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c080b2000 con 0x7f3c0809bfa0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c077fe640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 0x7f3c08152ac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00d5d0 tx=0x7f3bfc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004240 con 0x7f3c0809bfa0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3bfc0043e0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c080b22e0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003850 con 0x7f3c0809bfa0 2024-04-25T16:28:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.525+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c080b28a0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.526+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0806a180 con 0x7f3c0809bfa0 2024-04-25T16:28:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.527+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3bfc0039f0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.527+0000 7f3beffff640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bf40744b0 0x7f3bf40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.527+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f3bfc0957d0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.527+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bf40744b0 0x7f3bf40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.528+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bf40744b0 0x7f3bf40769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8034000 tx=0x7f3bf802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:54.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.532+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3bfc05fcc0 con 0x7f3c0809bfa0 2024-04-25T16:28:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:54.713+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3c080cc580 con 0x7f3c0809bfa0 2024-04-25T16:28:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.392+0000 7f3beffff640 1 -- 172.21.15.94:0/606962702 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f3bfc05f5c0 con 0x7f3c0809bfa0 2024-04-25T16:28:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bf40744b0 msgr2=0x7f3bf40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bf40744b0 0x7f3bf40769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8034000 tx=0x7f3bf802e000 comp rx=0 tx=0).stop 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 msgr2=0x7f3c08152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 0x7f3c08152ac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00d5d0 tx=0x7f3bfc00da90 comp rx=0 tx=0).stop 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 shutdown_connections 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f3bf40744b0 0x7f3bf40769c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c080cebb0 0x7f3c0806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c080afc00 0x7f3c080cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 --2- 172.21.15.94:0/606962702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c0809bfa0 0x7f3c08152ac0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.396+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 >> 172.21.15.94:0/606962702 conn(0x7f3c08074810 msgr2=0x7f3c08071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.397+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 shutdown_connections 2024-04-25T16:28:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.397+0000 7f3c0dd2e640 1 -- 172.21.15.94:0/606962702 wait complete. 2024-04-25T16:28:55.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T16:28:55.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.522+0000 7f7cac54e640 1 Processor -- start 2024-04-25T16:28:55.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.522+0000 7f7cac54e640 1 -- start start 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7cac54e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ca40b5490 0x7f7ca40b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7cac54e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7ca409c730 0x7f7ca40d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7cac54e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca409d0e0 con 0x7f7ca409c320 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7cac54e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca40b4a80 con 0x7f7ca409c730 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7cac54e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca409d3c0 con 0x7f7ca40b5490 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7ca409c730 0x7f7ca40d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7ca409c730 0x7f7ca40d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50578/0 (socket says 172.21.15.94:50578) 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 -- 172.21.15.94:0/2197380934 learned_addr learned my addr 172.21.15.94:0/2197380934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caa2ea640 1 --2- 172.21.15.94:0/2197380934 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40b5490 0x7f7ca40b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:55.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 -- 172.21.15.94:0/2197380934 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40b5490 msgr2=0x7f7ca40b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 --2- 172.21.15.94:0/2197380934 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40b5490 0x7f7ca40b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 -- 172.21.15.94:0/2197380934 >> v1:172.21.15.138:6789/0 conn(0x7f7ca409c320 legacy=0x7f7ca40d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.524+0000 7f7caaaeb640 1 -- 172.21.15.94:0/2197380934 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca40cf7b0 con 0x7f7ca409c730 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.525+0000 7f7caaaeb640 1 --2- 172.21.15.94:0/2197380934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca409c730 0x7f7ca40d2490 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40d1a50 tx=0x7f7c9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7316533bb10d3ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.525+0000 7f7ca92e8640 1 -- 172.21.15.94:0/2197380934 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c036070 con 0x7f7ca409c730 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.525+0000 7f7ca92e8640 1 -- 172.21.15.94:0/2197380934 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7c9c031600 con 0x7f7ca409c730 2024-04-25T16:28:55.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.525+0000 7f7ca92e8640 1 -- 172.21.15.94:0/2197380934 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0319f0 con 0x7f7ca409c730 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 -- 172.21.15.94:0/2197380934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca409c730 msgr2=0x7f7ca40d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 --2- 172.21.15.94:0/2197380934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca409c730 0x7f7ca40d2490 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40d1a50 tx=0x7f7c9c02a6a0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 -- 172.21.15.94:0/2197380934 shutdown_connections 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 --2- 172.21.15.94:0/2197380934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca409c730 0x7f7ca40d2490 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 --2- 172.21.15.94:0/2197380934 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40b5490 0x7f7ca40b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 -- 172.21.15.94:0/2197380934 >> 172.21.15.94:0/2197380934 conn(0x7f7ca40b5a90 msgr2=0x7f7ca40b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 -- 172.21.15.94:0/2197380934 shutdown_connections 2024-04-25T16:28:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.526+0000 7f7cac54e640 1 -- 172.21.15.94:0/2197380934 wait complete. 2024-04-25T16:28:55.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.528+0000 7f7cac54e640 1 Processor -- start 2024-04-25T16:28:55.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.528+0000 7f7cac54e640 1 -- start start 2024-04-25T16:28:55.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7cac54e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca409c320 0x7f7ca41d41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:55.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7cac54e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca409c730 0x7f7ca41d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:55.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7cac54e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 0x7f7ca406f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:55.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7cac54e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca40d29d0 con 0x7f7ca40b5490 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7cac54e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca40b4e10 con 0x7f7ca409c730 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7cac54e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca40d2b50 con 0x7f7ca409c320 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 0x7f7ca406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 0x7f7ca406f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50594/0 (socket says 172.21.15.94:50594) 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 -- 172.21.15.94:0/759086250 learned_addr learned my addr 172.21.15.94:0/759086250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caa2ea640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca409c320 0x7f7ca41d41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7ca9ae9640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca409c730 0x7f7ca41d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 -- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca409c320 msgr2=0x7f7ca41d41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca409c320 0x7f7ca41d41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 -- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca409c730 msgr2=0x7f7ca41d39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca409c730 0x7f7ca41d39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.529+0000 7f7caaaeb640 1 -- 172.21.15.94:0/759086250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca406eac0 con 0x7f7ca40b5490 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.530+0000 7f7caaaeb640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 0x7f7ca406f280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009f40 tx=0x7f7c9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.530+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c036050 con 0x7f7ca40b5490 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.530+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7c9c042070 con 0x7f7ca40b5490 2024-04-25T16:28:55.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.530+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca40cb310 con 0x7f7ca40b5490 2024-04-25T16:28:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.530+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009510 con 0x7f7ca40b5490 2024-04-25T16:28:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.530+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca40cb5f0 con 0x7f7ca40b5490 2024-04-25T16:28:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.531+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca406e8b0 con 0x7f7ca40b5490 2024-04-25T16:28:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.531+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c9c007c10 con 0x7f7ca40b5490 2024-04-25T16:28:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.532+0000 7f7c9affd640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7c880744e0 0x7f7c880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.532+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f7c9c0b5b30 con 0x7f7ca40b5490 2024-04-25T16:28:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.532+0000 7f7caa2ea640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7c880744e0 0x7f7c880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.532+0000 7f7caa2ea640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7c880744e0 0x7f7c880769f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40d3750 tx=0x7f7ca0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:55.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.536+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c9c04b020 con 0x7f7ca40b5490 2024-04-25T16:28:55.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:55.711+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7ca40cb070 con 0x7f7ca40b5490 2024-04-25T16:28:56.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.393+0000 7f7c9affd640 1 -- 172.21.15.94:0/759086250 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f7c9c080230 con 0x7f7ca40b5490 2024-04-25T16:28:56.393 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data_ec' removed 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.396+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7c880744e0 msgr2=0x7f7c880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.396+0000 7f7cac54e640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7c880744e0 0x7f7c880769f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40d3750 tx=0x7f7ca0009190 comp rx=0 tx=0).stop 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.396+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 msgr2=0x7f7ca406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.396+0000 7f7cac54e640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 0x7f7ca406f280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009f40 tx=0x7f7c9c004a40 comp rx=0 tx=0).stop 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.397+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 shutdown_connections 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.397+0000 7f7cac54e640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:6832/2607833826,v1:172.21.15.94:6833/2607833826] conn(0x7f7c880744e0 0x7f7c880769f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:56.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.397+0000 7f7cac54e640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ca40b5490 0x7f7ca406f280 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:56.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.397+0000 7f7cac54e640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca409c730 0x7f7ca41d39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:56.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.397+0000 7f7cac54e640 1 --2- 172.21.15.94:0/759086250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca409c320 0x7f7ca41d41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:56.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.397+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 >> 172.21.15.94:0/759086250 conn(0x7f7ca40b5a90 msgr2=0x7f7ca41e5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:56.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.398+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 shutdown_connections 2024-04-25T16:28:56.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:28:56.398+0000 7f7cac54e640 1 -- 172.21.15.94:0/759086250 wait complete. 2024-04-25T16:28:56.413 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T16:28:56.413 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T16:28:56.413 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.422 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T16:28:56.422 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T16:28:56.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.472 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T16:28:56.473 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T16:28:56.473 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.484 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T16:28:56.485 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T16:28:56.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.539 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T16:28:56.539 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T16:28:56.540 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T16:28:56.540 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.570 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T16:28:56.570 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T16:28:56.570 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.637 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T16:28:56.637 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T16:28:56.637 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.704 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T16:28:56.704 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T16:28:56.704 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.779 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T16:28:56.780 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T16:28:56.780 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.815 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T16:28:56.815 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T16:28:56.815 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.881 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T16:28:56.881 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T16:28:56.881 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:56.959 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T16:28:56.959 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T16:28:56.959 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:57.029 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T16:28:57.030 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T16:28:57.030 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T16:28:57.030 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:57.056 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T16:28:57.056 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T16:28:57.057 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:57.098 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T16:28:57.098 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T16:28:57.098 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T16:28:57.098 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:57.109 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T16:28:57.109 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T16:28:57.109 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:57.150 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T16:28:57.150 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T16:28:57.150 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:28:57.203 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T16:28:57.203 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T16:28:57.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T16:28:57.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:28:57.235 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T16:28:57.515 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:28:57.515 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T16:28:57.619 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:28:57.619 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T16:28:57.668 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:28:57.669 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T16:28:57.743 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:28:57.743 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T16:28:58.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:28:58.017 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T16:28:58.146 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:28:58.146 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T16:28:58.197 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:28:58.197 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T16:28:58.247 INFO:tasks.ceph:Archiving mon data... 2024-04-25T16:28:58.248 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/data/mon.a.tgz 2024-04-25T16:28:58.248 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-25T16:28:58.264 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.qZBZ2ujm58 2024-04-25T16:28:58.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.qZBZ2ujm58 2024-04-25T16:28:58.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.qZBZ2ujm58 2024-04-25T16:28:58.474 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.qZBZ2ujm58 is 281KB 2024-04-25T16:28:58.568 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.qZBZ2ujm58 2024-04-25T16:28:58.582 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/data/mon.b.tgz 2024-04-25T16:28:58.582 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-25T16:28:58.600 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.OtWVeY5TaV 2024-04-25T16:28:58.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OtWVeY5TaV 2024-04-25T16:28:58.753 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.OtWVeY5TaV 2024-04-25T16:28:58.835 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.OtWVeY5TaV is 377KB 2024-04-25T16:28:58.932 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.OtWVeY5TaV 2024-04-25T16:28:58.947 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/data/mon.c.tgz 2024-04-25T16:28:58.947 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-25T16:28:59.002 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.L893VF5t4R 2024-04-25T16:28:59.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.L893VF5t4R 2024-04-25T16:28:59.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.L893VF5t4R 2024-04-25T16:28:59.212 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.L893VF5t4R is 377KB 2024-04-25T16:28:59.307 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.L893VF5t4R 2024-04-25T16:28:59.321 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T16:28:59.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T16:28:59.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T16:28:59.403 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T16:28:59.439 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T16:28:59.439 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T16:28:59.440 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/remote/smithi094/crash 2024-04-25T16:28:59.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T16:28:59.467 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/remote/smithi138/crash 2024-04-25T16:28:59.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T16:28:59.501 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T16:28:59.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:59.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:59.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:59.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T16:28:59.645 INFO:tasks.ceph:Compressing logs... 2024-04-25T16:28:59.646 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:28:59.665 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:28:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44931.log 2024-04-25T16:28:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47207.log 2024-04-25T16:28:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42864.log 2024-04-25T16:28:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42320.log 2024-04-25T16:28:59.706 INFO:teuthology.orchestra.run.smithi094.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44931.log.gz 2024-04-25T16:28:59.706 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47429.log 2024-04-25T16:28:59.706 INFO:teuthology.orchestra.run.smithi094.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47207.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42320.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42864.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43035.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42320.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43035.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47429.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43398.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34344.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43035.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43398.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43398.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41657.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34344.log: /var/log/ceph/ceph-client.admin.43043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41926.log 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41657.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz 2024-04-25T16:28:59.707 INFO:teuthology.orchestra.run.smithi094.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.34344.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46527.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41926.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46527.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42052.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46527.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46997.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.41657.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42052.log.gz 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45626.log 2024-04-25T16:28:59.708 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45626.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45626.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46359.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46443.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46359.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46359.log.gz 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-04-25T16:28:59.709 INFO:teuthology.orchestra.run.smithi094.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46443.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45862.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42738.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45862.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40101.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42738.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45862.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47471.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40101.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42738.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47781.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40101.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45239.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47471.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46871.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.2.45239.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.47781.log.gz 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47081.log 2024-04-25T16:28:59.710 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45239.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46871.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47081.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47081.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40730.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.audit.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43038.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40730.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46072.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43038.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42570.log 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43038.log.gz 2024-04-25T16:28:59.711 INFO:teuthology.orchestra.run.smithi094.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46072.log.gz/var/log/ceph/ceph-osd.3.log: 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40600.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.42570.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42010.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45414.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42570.log.gz 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40600.log.gz 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45414.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42010.log.gz 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34711.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45414.log.gz 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43596.log: /var/log/ceph/ceph-client.admin.34711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46401.log 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34711.log.gz 2024-04-25T16:28:59.712 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46401.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40686.log 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46401.log.gz 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41884.log 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40686.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40686.log.gz 2024-04-25T16:28:59.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42822.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41884.log.gz 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44793.log 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47949.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.44793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44793.log.gz 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2024-04-25T16:28:59.714 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-04-25T16:28:59.746 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35157.log 2024-04-25T16:28:59.746 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T16:28:59.746 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35157.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T16:28:59.746 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35157.log.gz 2024-04-25T16:28:59.746 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35026.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5/var/log/ceph/ceph-client.admin.35026.log: --verbose -- /var/log/ceph/ceph-client.3.42410.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35066.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.35026.log.gz 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.3.42410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42410.log.gz 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T16:28:59.747 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35197.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34986.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42010.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35197.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.34986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35116.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.42010.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34986.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42010.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35237.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42015.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35116.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35237.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35237.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.42015.log: /var/log/ceph/ceph-mds.d.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42015.log.gz 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42414.log 2024-04-25T16:28:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35277.log 2024-04-25T16:28:59.749 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.3.42414.log: /var/log/ceph/ceph-mon.b.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42414.log.gz 2024-04-25T16:28:59.751 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35277.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35277.log.gz 2024-04-25T16:28:59.781 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40942.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40772.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40942.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45372.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40942.log.gz 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47039.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47039.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45372.log.gz 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40371.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47039.log.gz 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43932.log: -- /var/log/ceph/ceph-client.admin.40858.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40371.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40413.log 2024-04-25T16:28:59.782 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40371.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40858.log: /var/log/ceph/ceph-client.admin.40413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48009.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40858.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40413.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43397.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42696.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43397.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42362.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34608.log 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2024-04-25T16:28:59.783 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34608.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42362.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34608.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41068.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46030.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41068.log: /var/log/ceph/ceph-client.admin.47376.log: 89.0% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41068.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43039.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46030.log: /var/log/ceph/ceph-client.admin.43039.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46030.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43039.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42152.log: /var/log/ceph/ceph-client.admin.47165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gz 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gz 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40516.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40290.log 2024-04-25T16:28:59.784 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46787.log: /var/log/ceph/ceph-client.admin.40516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40516.log.gz 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48093.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45946.log: /var/log/ceph/ceph-client.admin.40290.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40290.log.gz 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41026.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45542.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48093.log: /var/log/ceph/ceph-client.admin.41026.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48093.log.gz 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41026.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42094.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45542.log: /var/log/ceph/ceph-client.admin.42094.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45542.log.gz 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47615.log 2024-04-25T16:28:59.785 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42094.log.gz 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44268.log 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47615.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47615.log.gz 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47657.log 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45235.log 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44268.log: /var/log/ceph/ceph-client.admin.47657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44268.log.gz 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40900.log 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47657.log.gz 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.2.45235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46913.log 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45235.log.gz 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-04-25T16:28:59.786 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40900.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42486.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46913.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40194.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42486.log: /var/log/ceph/ceph-client.admin.42194.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42486.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46149.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr: 88.6%/var/log/ceph/ceph-client.admin.47823.log: -- replaced with /var/log/ceph/ceph-client.admin.40194.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44798.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46317.log 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46149.log.gz 2024-04-25T16:28:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.44798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44798.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46955.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46317.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43680.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28316.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46955.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46955.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43764.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45752.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.tmp-client.admin.28316.log: /var/log/ceph/ceph-client.admin.43764.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28316.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43764.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43394.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40984.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45752.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43394.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43394.log.gz 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46233.log 2024-04-25T16:28:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45988.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40984.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46233.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46233.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41110.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47123.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45988.log: /var/log/ceph/ceph-client.admin.41110.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45988.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41110.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47123.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47123.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44184.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41152.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41152.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44184.log.gz 2024-04-25T16:28:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46829.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41152.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45584.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47291.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44100.log: /var/log/ceph/ceph-client.admin.45584.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45584.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42278.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47291.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47291.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42444.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42278.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42444.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40642.log 2024-04-25T16:28:59.790 INFO:teuthology.orchestra.run.smithi094.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42444.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43399.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40642.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40642.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43890.log: /var/log/ceph/ceph-client.admin.43399.log: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43399.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41218.log: /var/log/ceph/ceph-client.admin.46191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46569.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47333.log 2024-04-25T16:28:59.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47333.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47333.log.gz 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47573.log: /var/log/ceph/ceph-mds.a.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43036.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42528.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43036.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44310.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43036.log.gz 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46485.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42528.log: /var/log/ceph/ceph-client.admin.44310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-04-25T16:28:59.792 INFO:teuthology.orchestra.run.smithi094.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42528.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47249.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46485.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44310.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40814.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43279.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47249.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40456.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48051.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40814.log: /var/log/ceph/ceph-client.admin.40456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48051.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40814.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40456.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40558.log 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48051.log.gz 2024-04-25T16:28:59.793 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47531.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46611.log: /var/log/ceph/ceph-client.admin.40558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47699.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47531.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40558.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47531.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44474.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43046.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47699.log: gzip/var/log/ceph/ceph-client.admin.44474.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47699.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44474.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45668.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44016.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43046.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-04-25T16:28:59.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45668.log: /var/log/ceph/ceph-client.admin.42236.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45668.log.gz 2024-04-25T16:28:59.795 INFO:teuthology.orchestra.run.smithi094.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-04-25T16:28:59.795 INFO:teuthology.orchestra.run.smithi138.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T16:28:59.799 INFO:teuthology.orchestra.run.smithi138.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T16:28:59.815 INFO:teuthology.orchestra.run.smithi138.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T16:28:59.871 INFO:teuthology.orchestra.run.smithi094.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T16:29:00.328 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T16:29:00.338 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T16:29:00.344 INFO:teuthology.orchestra.run.smithi138.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T16:29:00.346 INFO:teuthology.orchestra.run.smithi138.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T16:29:00.348 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T16:29:00.350 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T16:29:00.378 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T16:29:00.380 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T16:29:00.380 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m0.665s 2024-04-25T16:29:00.380 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m2.622s 2024-04-25T16:29:00.380 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.114s 2024-04-25T16:29:00.393 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T16:29:00.395 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T16:29:00.395 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.718s 2024-04-25T16:29:00.395 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m2.796s 2024-04-25T16:29:00.395 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.211s 2024-04-25T16:29:00.395 INFO:tasks.ceph:Archiving logs... 2024-04-25T16:29:00.396 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/remote/smithi094/log 2024-04-25T16:29:00.415 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T16:29:00.965 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/remote/smithi138/log 2024-04-25T16:29:00.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T16:29:01.361 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T16:29:01.370 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T16:29:01.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T16:29:01.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T16:29:01.446 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T16:29:01.455 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T16:29:01.455 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:29:01.458 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:29:01.470 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:29:01.471 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:29:01.473 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:29:01.473 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T16:29:01.473 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 19 +1105us[+1105us] +/- 64ms 2024-04-25T16:29:01.473 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -805us[ -810us] +/- 55ms 2024-04-25T16:29:01.474 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -189us[ -194us] +/- 79ms 2024-04-25T16:29:01.474 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:29:01.476 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:29:01.476 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T16:29:01.476 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 18 +1128us[+1128us] +/- 64ms 2024-04-25T16:29:01.476 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -783us[ -775us] +/- 55ms 2024-04-25T16:29:01.476 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -172us[ -172us] +/- 79ms 2024-04-25T16:29:01.476 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:29:01.477 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T16:29:01.514 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T16:29:01.515 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T16:29:01.563 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T16:29:01.627 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T16:29:01.685 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:29:01.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:29:01.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:29:01.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:29:01.738 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T16:29:01.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T16:29:01.757 INFO:teuthology.task.internal:Duration was 492.168370 seconds 2024-04-25T16:29:01.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T16:29:01.766 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T16:29:01.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T16:29:01.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T16:29:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:29:01.814 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:29:02.138 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T16:29:02.138 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:29:02.139 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T16:29:02.199 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:29:02.199 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T16:29:02.226 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T16:29:02.226 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T16:29:02.241 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T16:29:02.290 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T16:29:02.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T16:29:02.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T16:29:02.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T16:29:02.437 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T16:29:02.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T16:29:02.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T16:29:02.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T16:29:02.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:29:02.503 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:29:02.522 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-25T16:29:02.546 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-25T16:29:02.559 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:29:02.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:29:02.588 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:29:02.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:29:02.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T16:29:02.623 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T16:29:02.623 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/remote/smithi094 2024-04-25T16:29:02.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T16:29:02.661 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673262/remote/smithi138 2024-04-25T16:29:02.661 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T16:29:02.693 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T16:29:02.693 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T16:29:02.698 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T16:29:02.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T16:29:02.754 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T16:29:02.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T16:29:02.781 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T16:29:02.782 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T16:29:02.784 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T16:29:02.796 INFO:teuthology.orchestra.run.smithi094.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 16:29 /home/ubuntu/cephtest 2024-04-25T16:29:02.798 INFO:teuthology.orchestra.run.smithi138.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 16:29 /home/ubuntu/cephtest 2024-04-25T16:29:02.799 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T16:29:02.809 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T16:29:02.911 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/truncate_delay} duration: 492.1683704853058 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T16:29:02.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T16:29:02.981 INFO:teuthology.run:pass